MNRAS

000 , 1-21 (2024)

## An ALMA survey of submillimetre galaxies in the Extended Chandra Deep Field South: an unbiased study of SMG environments measured with narrowband imaging

Thomas M. Cornish 1 2 , ★ , Julie Wardlow 1 † , Heather Wade 1 , David Sobral 1 3 , W. N. Brandt 4 5 6 , , , , Pierre Cox 7 , Helmut Dannerbauer 8 9 , Roberto Decarli 10 , Bitten Gullberg 11 12 , Kirsten Knudsen 13 , , , John Stott 1 , Mark Swinbank 14 , Fabian Walter 15 , Paul van der Werf 16

1 Department of Physics, Lancaster University, Lancaster, LA1 4YB, UK

2 Department of Physics, University of Oxford, Denys Wilkinson Building, Keble Road, Oxford, OX1 3RH, UK

- 3 BNP Paribas Corporate &amp; Institutional Banking, Lisbon, Portugal
- 4 Department of Astronomy &amp; Astrophysics, The Pennsylvania State University, 525 Davey Lab, University Park, PA 16802, USA
- 5 Institute for Gravitation and the Cosmos, The Pennsylvania State University, University Park, PA 16802, USA
- 6 Department of Physics, The Pennsylvania State University, University Park, PA 16802, USA
- 7 Sorbonne Université, CNRS UMR 7095, Institut d'Astrophysique de Paris, 98bis bvd Arago, 75014 Paris, France
- 8 Instituto de Astrofísica de Canarias (IAC), E-38205 La Laguna, Tenerife, Spain
- 9 Universidad de La Laguna, Dpto. Astrofísica, E-38206 La Laguna, Tenerife, Spain
- 10 INAF - Osservatorio di Astrofisica e Scienza dello Spazio di Bologna, Via Gobetti 93/3, 40129 Bologna, Italy
- 11 DTU Space, Technical University of Denmark, Elektrovej 327, 2800 Kgs. Lyngby, Denmark
- 12 Cosmic Dawn Centre (DAWN), Copenhagen, Denmark
- 13 Department of Space, Earth and Environment, Chalmers University of Technology, SE-412 96 Gothenburg, Sweden
- 14 Centre for Extragalactic Astronomy, Department of Physics, Durham University, South Road, Durham DH1 3LE, UK
- 15 Max-Planck-Institut für Astronomy, Königstuhl 17, D-69117, Heidelberg, Germany
- 16 Leiden Observatory, Leiden University, P.O. Box 9513, 2300 RA Leiden, The Netherlands

Accepted XXX. Received YYY; in original form ZZZ

## ABSTRACT

Submillimetre galaxies (SMGs) are some of the most extreme star-forming systems in the Universe, whose place in the framework of galaxy evolution is as yet uncertain. It has been hypothesised that SMGs are progenitors of local early-type galaxies, requiring that SMGs generally reside in galaxy cluster progenitors at high redshift. We test this hypothesis and explore SMG environments using a narrowband VLT/HAWK-I+GRAAL study of H 𝛼 and [Oiii] emitters around an unbiased sample of three ALMA-identified and spectroscopically-confirmed SMGs at 𝑧 ∼ 2 3 and . 𝑧 ∼ 3 3, where these SMGs were selected . solely on spectroscopic redshift. Comparing with blank-field observations at similar epochs, we find that one of the three SMGs lies in an overdensity of emission-line sources on the ∼ 4 Mpc scale of the HAWK-I field of view, with overdensity parameter 𝛿 𝑔 = 2 6 . + 1 4 . -1 2 . . A second SMG is significantly overdense only on ≲ 1 6 Mpc scales and the final SMG is consistent with residing . in a blank field environment. The total masses of the two overdensities are estimated to be log ( 𝑀 ℎ / M ⊙) = 12.1-14.4, leading to present-day masses of log ( 𝑀 ℎ,𝑧 = 0 / M ⊙) = 12.9-15.9. These results imply that SMGs occupy a range of environments, from overdense protoclusters or protogroups to the blank field, suggesting that while some SMGs are strong candidates for the progenitors of massive elliptical galaxies in clusters, this may not be their only possible evolutionary pathway.

Key words: galaxies: star formation - galaxies: photometry - galaxies: evolution - submillimetre: galaxies

## 1 INTRODUCTION

Since their discovery more than two decades ago, submillimetre galaxies (SMGs; e.g. Smail et al. 1997; Barger et al. 1998; Hughes et al. 1998; Eales et al. 1999; Blain et al. 2002; Coppin et al. 2006; Casey et al. 2014; Hodge &amp; da Cunha 2020) have proven to be important laboratories for exploring galaxy formation and evolution. These

- ★ E-mail: thomas.cornish@physics.ox.ac.uk

galaxies are identified in (sub)millimetre surveys and have typical infrared (IR) luminosities of 𝐿 IR ∼ 10 12 -13 𝐿 ⊙ corresponding to star formation rates of ∼ 10 2 -3 M yr ⊙ -1 (e.g. Chapman et al. 2005; Pope et al. 2006; Wardlow et al. 2011; Magnelli et al. 2012; Swinbank et al. 2014; MacKenzie et al. 2017; Michałowski et al. 2017; RowanRobinson et al. 2018; Cheng et al. 2019; Greenslade et al. 2020). They are massive, with stellar masses of 𝑀 ∗ ∼ 10 10 -11 M ⊙ (e.g. Hainline et al. 2011; Michałowski et al. 2012; Gruppioni et al. 2013; Simpson et al. 2014; da Cunha et al. 2015; Dudzeviči¯t˙ e et al. 2020; u Pantoni et al. 2021), dust masses of ≳ 10 8 M (e.g. Clements et al.

- † E-mail: j.wardlow@lancaster.ac.uk

⊙

2010; Miettinen et al. 2017; Dudzeviči¯t˙ e et al. 2020; Pantoni et al. u 2021), and cold gas masses of ∼ 10 11 M ⊙ (e.g. Greve et al. 2005; Tacconi et al. 2006; Bothwell et al. 2013; Birkin et al. 2021), but with gas depletion times of just a few hundred Myr (e.g. Tacconi et al. 2006; Birkin et al. 2021). The SMG redshift distribution peaks at 𝑧 ∼ 2 5 (e.g. . Chapman et al. 2005; Wardlow et al. 2011; Koprowski et al. 2014; Danielson et al. 2017; Smith et al. 2017; Stach et al. 2019; Dudzeviči¯t˙ e et al. 2020; da Cunha et al. 2021), making u these massive dusty galaxies the most intense star-forming systems in the Universe during its peak epoch of star formation (Madau &amp; Dickinson 2014). SMGs contribute up to ∼ 20 percent of the cosmic star-formation rate density at 𝑧 ∼ 2 (e.g. Coppin et al. 2006; Barger et al. 2012; Swinbank et al. 2014).

Theextreme properties of SMGs has long made them a good test of galaxy evolution models (e.g. Baugh et al. 2005; Lacey et al. 2008, 2010; Davé et al. 2010; Narayanan et al. 2010, 2015; Béthermin et al. 2011; Niemi et al. 2012; Hayward et al. 2021; Lovell et al. 2021), yet questions about their evolution and role in the evolution of other galaxies remain. SMGs have similar properties to those expected of the progenitors of local massive elliptical galaxies, which formed most of their stars in short bursts at 𝑧 ≳ 2 (Ellis et al. 1997; Blakeslee et al. 2003). Indeed, the dust emission from SMGs is typically compact (e.g. Hodge et al. 2016; Gullberg et al. 2019), which is consistent with a scenario in which a gas-rich 𝑧 ≳ 2 galaxy undergoes a compact starburst, leading to a compact quiescent galaxy, which eventually evolves into a local elliptical galaxy (Toft et al. 2014; Simpson et al. 2014; Ikarashi et al. 2015). Since local ellipticals are predominantly found in galaxy clusters (e.g. Dressler 1980) then if SMGs are indeed a progenitor phase in their formation, then it is expected that SMGs should reside in early galaxy clusters, or 'protoclusters', at 𝑧 ≳ 2.

Galaxy protoclusters (for a review, see Overzier 2016) are typically defined as structures that will collapse and virialise to form a galaxy cluster by 𝑧 = 0. Simulations have shown that in a Λ CDM universe, protoclusters form hierarchically at the highest density regions of the matter distribution in the universe (the 'cosmic web'; Bond et al. 1996) at 𝑧 ∼ 4-6 (e.g. Baugh et al. 1998; De Lucia &amp; Blaizot 2007). As such, protoclusters are characterised by overdensities of galaxies relative to the average galaxy density in the coeval blank field. Conversely to their present-day descendants, galaxies in a protocluster are generally not bound to a single halo; they instead occupy large structures extended over megaparsec (Mpc) scales, with the main halo containing as little as 20 percent of the member galaxies (e.g. Chiang et al. 2013; Muldrew et al. 2015).

Unfortunately, observationally identifying protoclusters is challenging. Methods of detecting galaxy clusters from their X-ray emission (e.g. Trümper 1993; Böhringer et al. 2001; Henry et al. 2006; Pacaud et al. 2016) or by searching for their imprint on the cosmic microwave background at millimetre wavelengths (e.g. Staniszewski et al. 2009; Williamson et al. 2011; Hasselfield et al. 2013; Bleem et al. 2015; Planck Collaboration et al. 2016) via the SunyaevZel'dovich effect (Sunyaev &amp; Zeldovich 1972) are rendered impractical due to the lack of a hot intracluster medium (ICM). Similarly, searches for high concentrations of passive early-type galaxies occupying a tight 'red sequence' in colour-magnitude space (e.g. Gladders &amp; Yee 2000, 2005; Muzzin et al. 2009; Wilson et al. 2009; Gilbank et al. 2011) become ineffective since the stellar populations of galaxies in protoclusters typically have not evolved sufficiently for a significant 4000 Å break to be detected. Consequently, the majority of protocluster surveys resort to searching for overdensities of galaxies at high redshift. Such searches depend on the existence of accurate redshift information across large cosmological volumes, and several protoclusters have been discovered serendipitously through large spectroscopic surveys (e.g. Steidel et al. 1998, 2000, 2005; Cucciati et al. 2014; Lemaux et al. 2014). In lieu of expensive large-scale spectroscopic observations, an alternative method is to use wide-field narrowband photometric surveys to search for overdensities of galaxies with strong emission lines at a particular observed-frame wavelength (e.g. Ly 𝛼 or H 𝛼 emitters; Venemans et al. 2002; Shimasaku et al. 2003; Matsuda et al. 2004; Palunas et al. 2004; Venemans et al. 2005; Hatch et al. 2011; Kuiper et al. 2011b; Matsuda et al. 2011; Tanaka et al. 2011; Hayashi et al. 2012; Koyama et al. 2013; Zheng et al. 2021).

Whether SMGs commonly inhabit protoclusters or protoclusterlike environments is as yet uncertain. Several examples of SMGs residing in protoclusters have been documented (e.g. Ivison et al. 2000; Smail et al. 2003; Geach et al. 2005; Daddi et al. 2009; Matsuda et al. 2011; Ivison et al. 2013; Casey et al. 2015), but these systems were selected for detailed follow-up because of prior evidence of high galaxy densities. I.e. they comprise a highly biased subset and therefore cannot be used to make inferences about the general SMG population.

Clustering studies have been used to obtain statistical measurements indicative of the whole SMG population. Results from singledish clustering measurements suggest that on average SMGs at 𝑧 ∼ 2 5 reside in dark matter halos of mass . ∼ 10 13 M ⊙ (e.g. Hickox et al. 2012; Wilkinson et al. 2017). This is marginally lower than expected for the progenitors of massive ellipticals, and implies that SMGsareinstead more likely to evolve into 2-3 𝐿 ∗ galaxies in groups and small clusters. However, these halo mass measurements have typical uncertainties of ∼ 0 5 dex due to the difficulties associated . with obtaining accurate photometric redshifts for SMGs. Furthermore, these clustering measurements relied on the statistical identification of optical/near-IR counterparts to submillimetre sources detected in low-resolution single-dish surveys, which are incorrect in ∼ 30 percent of cases and incomplete in a further ∼ 30 percent (e.g. Hodge et al. 2013a). More recently, García-Vergara et al. (2020) and Stach et al. (2021) measured the clustering of SMGs which had been followed up interferometrically with the Atacama Large Millimetre/submillimetre Array (ALMA). Using a small sample of 50 ALMA-identified SMGs with spectroscopic redshifts and employing forward modelling to correct for incompleteness, García-Vergara et al. (2020) estimated halo masses that are a factor of ∼ 3 8 lower . than other studies of SMGs. From a significantly larger parent sample, Stach et al. (2021) selected a complete sample of ∼ 350 ALMAidentified SMGs with photometric redshifts to measure halo masses consistent with the results from the single-dish studies, particularly at 𝑧 &gt; 2 (Hickox et al. 2012; Wilkinson et al. 2017). Overall, the picture from clustering measurements is complex, and differing results from different studies may be methodological, due to sample selection or cosmic variance. Other ways of measuring the environments of SMGs are required.

Statistical photometric redshifts have identified galaxy overdensities around ∼ 5-60 percent of SMGs (e.g. Davies et al. 2014; Simpson et al. 2014; Smolčić et al. 2017), but these are subject to significant selection biases (e.g. see Section 6 in Smolčić et al. 2017), and few overdensities have been spectroscopically confirmed. Instead, in order to determine the nature of a 'typical' SMG environment, and thus confirm whether SMGs really are the progenitors of massive elliptical galaxies in local clusters, we need targeted observations of individual SMGs, but with no prior knowledge of their environments in order to avoid biases. To this end, we have conducted a wide-field narrowband survey of the environments of three SMGs identified as part of the ALESS project (Hodge et al. 2013a), in which follow-

up observations of submillimetre sources detected in the LABOCA ECDFS Submillimetre Survey (LESS; Weiß et al. 2009) were conducted using ALMA. We search for overdensities of H 𝛼 or [Oiii] emitters around these three SMGs to assess whether they reside in protocluster-like environments. The target SMGs were selected on the basis of redshift only, and with no prior information about their environments. Our method is similar to that employed by Matsuda et al. (2011), which combined narrowband photometry with submillimetre observations to identify an overdensity of H 𝛼 emitters around two SMGs in SSA 13. However, the SMGs targeted by Matsuda et al. (2011) were already known to be closely grouped with each other and three optically-faint radio-galaxies. Our study is the first to perform such an analysis around a sample of SMGs that are selected without prior knowledge of their environment.

The structure of this paper is as follows: in §2 we describe the SMG sample selection, our observations and data reduction, and the identification of candidate companion galaxies for each target SMG; §3 includes our main results, analysis and discussion; our conclusions are presented in §4. Throughout this paper we use a Λ CDM cosmology with Ω M = 0 315, . ΩΛ = 0 685 and . 𝐻 0 = 67 4 . km s -1 Mpc -1 (Planck Collaboration 2020). All magnitudes are presented in the AB system, where a 1 𝜇 Jy source has a magnitude of 23 9 (Oke &amp; Gunn 1983). .

## 2 OBSERVATIONS AND GALAXY IDENTIFICATION

In this study we use the High Acuity Wide-field K-band Imager (HAWK-I; Pirard et al. 2004; Casali et al. 2006; Kissler-Patig et al. 2008; Siebenmorgen et al. 2011) on the Very Large Telescope (VLT) to investigate the environments of three ALMA-identified SMGs from ALESS. As part of ALESS, extensive follow-up was undertaken to obtain spectroscopic redshifts of the SMGs (Danielson et al. 2017; Birkin et al. 2021), which enables a search for galaxies that share environments with these submillimetre sources. The wide-field imaging capabilities and narrowband photometric filters of HAWKI allow for an efficient emission-line survey of their environments, which are expected to span physical scales on the order of several Mpc if consistent with being protoclusters (e.g. Chiang et al. 2013; Muldrew et al. 2015; Yajima et al. 2022).

## 2.1 Sample selection

The blank-field LABOCA ECDFS Submillimetre Survey (LESS) observed 0 5 . × 0 5 degrees in ECDFS with APEX/LABOCA and . detected 126 sources at &gt; . 3 7 𝜎 at 870𝜇 m(Weiß et al. 2009). Each of these sources was followed-up with ALMA to yield the 131 ALESS sources described in Hodge et al. (2013a), divided into a main catalogue of 99 SMGs and a supplementary catalogue of 32 SMGs. The SMGs in the main catalogue all lay within the ALMA primary beam full width at half-maximum (FWHM) of the highest-quality maps, while those in the supplementary catalogue were either extracted from outside the primary beam or from lower-quality maps (Hodge et al. 2013a).

Spectroscopic redshifts were obtained for 52 of the 131 ALESS SMGs by Danielson et al. (2017), and targets for our study are selected from these 52 ALMA-identified and spectroscopicallyconfirmed SMGs. We require SMGs with spectroscopic redshifts that shift the H 𝛼 or [Oiii] 𝜆 5007 emission lines into the wavelength coverage of the HAWK-I Br 𝛾 filter; this requires that the SMGs are located at 𝑧 = 2 299 . ± 0 023 or . 𝑧 = 3 324 . ± 0 060. Of the 52 ALMA-. identified SMGs with spectroscopic redshifts from Danielson et al.

Table 1. Details of each of the three SMGs included in our sample.

| SMG         | 𝑧 spec    | 𝑆 870 [mJy] 𝑎   | mag 𝐾 𝑠 𝑏        | Target line 𝑐   |
|-------------|-----------|-----------------|------------------|-----------------|
| ALESS 5.1   | 3 . 303 𝑑 | 7 . 8 ± 0 . 7   | 19 . 79 ± 0 . 01 | [Oiii] 𝜆 5007   |
| ALESS 75.2  | 2 . 294 𝑒 | 5 . 0 ± 1 . 2   | 20 . 67 ± 0 . 01 | H 𝛼             |
| ALESS 102.1 | 2 . 296 𝑓 | 3 . 1 ± 0 . 5   | 21 . 07 ± 0 . 08 | H 𝛼             |

𝑎 Primary-beam-corrected ALMA 870 𝜇 mflux densities from Hodge et al. (2013a).

𝑏 From Simpson et al. (2014).

𝑐 The emission line used in this study to identify companion galaxies for each SMG, exploiting the fact that these lines shift into the wavelength coverage of the HAWK-I Br 𝛾 filter at the redshifts of the SMGs (see §2.1). 𝑑 Obtained via detection of the CO(4-3) and [Cii] emission lines (Birkin et al. 2021).

- 𝑒 Based on H 𝛼 +[Nii] and [Sii] detections (Danielson et al. 2017).
- 𝑓 Determined using a combination of Ly 𝛼 , Ciii] and continuum measurements (Danielson et al. 2017).

(2017), five (ALESS 6.1, 75.2, 87.1, 102.1, and 112.1) have spectroscopic redshifts within the desired range for H 𝛼 . A sixth SMG (ALESS5.1) has a CO-derived spectroscopic redshift of 𝑧 = 3 303 . (Birkin et al. 2021), which places [Oiii] in the Br 𝛾 coverage.

These six SMGs were the proposed targets for observations in four HAWK-Ipointings(PID:0103.A-0668). The six SMGs were selected purely based on their spectroscopic redshifts, with no consideration of their environments. Of the four pointings, only two were observed during the service-mode observations and the choice of pointings was random. The two observed pointings contain three of the six proposed targets: ALESS 5.1, ALESS 75.2, and ALESS 102.1, whose spectroscopic redshifts are 𝑧 = 3 303, 2.294, and 2.296, respectively. Details . of these three targeted SMGs are provided in Table 1. A total of 16 other ALESS SMGs lie within the two HAWK-I pointings, but these are not considered in this study as their redshifts are such that no bright emission lines are expected in the narrowband filter. Indeed, those that are detected in our HAWK-I observations fail to meet our criteria for being emission-line galaxies (see §2.4 and Figure 1).

## 2.2 HAWK-I data

Eachpointing was imaged using the HAWK-I instrument (Pirard et al. 2004; Casali et al. 2006; Kissler-Patig et al. 2008; Siebenmorgen et al. 2011) on the VLT in the 𝐾 𝑠 (central wavelength: 𝜆 𝑐 = 2 146 . 𝜇 m; FWHM: Δ 𝜆 = 0 324 . 𝜇 m) and Br 𝛾 ( 𝜆 𝑐 = 2 165 . 𝜇 m; Δ 𝜆 = 0 030 . 𝜇 m) filters (Kissler-Patig et al. 2008). The FWHM of the Br 𝛾 filter is equivalent to Δ 𝑧 = 0 046 (0.060) at . 𝑧 ∼ 2 3 (3.3), corresponding to . a velocity width of Δ 𝑣 = 4200 km s -1 . All three observing blocks (OBs) for the field containing ALESS 5.1 and 75.2 (hereafter Pointing 5+75) were executed on 2019 August 21, while the OBs for the field containing ALESS 102.1 (hereafter Pointing 102) were split among three separate nights from 2019 August 21 to 2020 January 01. The total exposure times for Pointing 5+75 (Pointing 102) were 7.2 ks (6.6 ks) and 900 s (660 s) in the Br 𝛾 and 𝐾 𝑠 filters, respectively. Individual exposures of 120 s (Br 𝛾 ) and 30 s ( 𝐾 𝑠 ) were taken using the 'HAWKI img obs AutoJitter' template, with five random dither positions within a 20 ′′ box for each filter in each OB. Each pointing covers a 7 . ′ 5 × 7 5 area, except for a cross-shaped gap of width . ′ 15 ′′ between the detector's four 2k × 2k Hawaii 2RG arrays. Using HAWK-I's GRAAL system (GRound layer Adaptive optics system Assisted by Lasers; Arsenault et al. 2008; Paufique et al. 2010), we

achieved point spread functions (PSFs) of ∼ ′′ 0 . 4 in Br 𝛾 and ∼ ′′ 0 . 3 in 𝐾 𝑠 (see Table 2).

The data were reduced using a custom Python-based pipeline, with each of the four detector chips treated separately. Briefly, the pipeline begins by dark-subtracting the data and subsequently using twilight flats to perform flat-fielding. We then use SExtractor (Bertin &amp; Arnouts 1996) to detect sources in each of the flattened frames and produce individual masks. A final flat field is produced for each frame by median combining all masked frames from the same OB except the frame being flattened; the frames are then flattened using their unique final flat fields. The astrometry of each flattened frame is then calibrated by using scamp (Bertin 2006) to match our detected sources with sources detected in a reference 𝐾 -band image, correcting for any distortions across the field of view by fitting a third-order polynomial. The reference images used for Pointing 5+75 and Pointing 102 were taken from the Taiwan ECDFS NIR Survey (TENIS; Hsieh et al. 2012) and the MUltiwavelength Survey by Yale-Chile (MUSYC; Gawiser et al. 2006b; Taylor et al. 2009), respectively. Different reference images were required for each pointing because while TENIS is deeper and has higher resolution, roughly a quarter of the Pointing 102 field of view lies outside of the TENIS coverage. Finally, the astrometrically-corrected frames were median combined using SWarp (Bertin 2010). The resultant stacks in both bands were photometrically calibrated using MUSYC 𝐾 𝑠 data (Taylor et al. 2009; Simpson et al. 2014) such that they all had a zeropoint magnitude of 30.0 mag and ensure a median ( 𝐾 𝑠 - Br 𝛾 ) colour of 0.

Source detection and photometry were conducted using SExtractor (Bertin &amp; Arnouts 1996) operating in dual-image mode; the stacked Br 𝛾 images were used to identify the positions of sources, and then photometry was extracted at these positions in both the Br 𝛾 and 𝐾 𝑠 images to ensure that any difference between the measured Br 𝛾 and 𝐾 𝑠 photometry is a purely intrinsic property of the sources and not caused by positional offsets. After masking noisy regions near the edges of the stacked images, we detected a total of 2175 sources in Pointing 5+75 and 1754 in Pointing 102. Apertures with a diameter of 1 . ′′ 25 were used, as this is large enough to contain the majority of the flux for all detected sources while minimising the amount of additional background noise captured. The 3 𝜎 limiting magnitudes measured in these 1 . ′′ 25 diameter apertures are provided for each filter and each HAWK-I detector chip in Table 2. To account for the variation in size of the detected sources, we then estimated total magnitudes in each filter by selecting all bright ( 𝑚 𝐾 𝑠 &lt; 19 5) . sources and (for each pointing separately) calculating the median difference between their fixed-aperture magnitudes and their magnitudes measured by SExtractor in adaptively-scaled (Kron 1980) apertures (mag\_auto; Bertin &amp; Arnouts 1996); this difference was then added to the fixed-aperture magnitudes of all sources in the pointing to obtain their total magnitudes. The choice to use the same aperture size for all sources and apply a correction (as opposed to simply using the mag\_auto values) ensures that estimates of the total magnitudes are self-consistent whilst also closely matching existing photometry in the same band.

## 2.3 Ancillary data

There exists a wealth of photometric data in the ECDFS, which supplements our HAWK-I photometry. Archival TENIS (Hsieh et al. 2012), MUSYC (Taylor et al. 2009) and HAWK-I (Zibetti, priv. comm.) 𝐾 𝑠 data were collated by Simpson et al. (2014) (hereafter S14) and then used to calibrate our astrometry and photometry (see §2.2).

In §2.5 and §3.4 we fit spectral energy distributions to galaxies

Table 2. Limiting 3 𝜎 ABmagnitudes and resolution for each stacked image. Limiting magnitudes are measured using randomly-placed 1 . ′′ 25 diameter apertures. Due to variation in the four HAWK-I detector chips, each quadrant is considered separately.

| Pointing      | Quadrant 𝑎   | 𝑚 3 𝜎 lim   | 𝑚 3 𝜎 lim   | PSF ( ′′ )   | PSF ( ′′ )   |
|---------------|--------------|-------------|-------------|--------------|--------------|
|               |              | Br 𝛾        | 𝐾 𝑠         | Br 𝛾         | 𝐾 𝑠          |
| Pointing 5+75 | Q1           | 24.29       | 24.01       | 0.28         | 0.27         |
| Pointing 5+75 | Q2           | 24.19       | 24.27       | 0.27         | 0.27         |
| Pointing 5+75 | Q3           | 24.30       | 24.19       | 0.35         | 0.26         |
| Pointing 5+75 | Q4           | 24.22       | 24.27       | 0.32         | 0.26         |
| Pointing 102  | Q1           | 24.09       | 23.85       | 0.38         | 0.30         |
| Pointing 102  | Q2           | 24.08       | 24.00       | 0.37         | 0.30         |
| Pointing 102  | Q3           | 24.19       | 24.01       | 0.44         | 0.30         |
| Pointing 102  | Q4           | 24.11       | 24.09       | 0.41         | 0.29         |

𝑎 Quadrants are assigned the same labels as in Kissler-Patig et al. (2008).

in our sample in order to first derive properties such as photometric redshift, stellar mass and star formation rate. For the photometric redshifts we use eazy-py 1 - an updated version of the photometric redshift code eazy (Brammer et al. 2008) written in Python (see §2.5) - while for the other galaxy properties we use magphys (da Cunha et al. 2008, see §3.4). To this end, we also make use of existing ECDFS images spanning the ultraviolet (UV) to midinfrared (MIR; see §2.5). These images were sourced either from the MUSYC 2010 Public Data Release (Cardamone et al. 2010) or from TENIS (Hsieh et al. 2012). The MUSYC dataset consists of 𝑈𝑈 38 𝐵𝑉𝑅𝐼 broadband images from the Wide Field Imager (WFI) on the MPG/ESO 2.2m telescope (Hildebrandt et al. 2006); 5000 Å narrowband and 𝑧 ′ broadband imaging from the Mosaic-II camera on the CTIO Blanco 4m telescope (Gawiser et al. 2006b,a), with 𝐽𝐾 𝑠 broadband imaging from the Infrared Sideport Imager on the same telescope (Taylor et al. 2009); 18 medium-band (IA427, IA445, IA464, IA484, IA505, IA527, IA550, IA574, IA598, IA624, IA651, IA679, IA709, IA738, IA767, IA797, IA827, IA856) images taken with the Subaru telescope's Suprime-Cam (Cardamone et al. 2010); Spitzer /IRAC images at 3.6, 4.5, 5.8 and 8.0 𝜇 m (Cardamone et al. 2010; Damen et al. 2011) 2 .

We also make use of spectroscopic redshifts in the ECDFS from studies whose areas overlap with our pointings, obtained from publicly-available composite catalogues 3 4 , (Silverman et al. 2010). The spectroscopic redshifts used are from the VIMOS VLT Deep Survey (VVDS; Le Fèvre et al. 2005); the GOODS/VIMOS survey (Popesso et al. 2009; Balestra et al. 2010); the Extended Chandra Deep Field-South Survey (Silverman et al. 2010); and Treister et al. (2009). Additionally we utilise the results of the spectroscopic study conducted as part of ALESS by Danielson et al. (2017).

Finally, we make use of the Lehmer et al. (2005) Chandra point source catalogue for the identification of AGN in our final sample (see §2.5).

1 https://github.com/gbrammer/eazy-py

- 2 𝐻 -band data are also available but our pointings are not covered.
- 3 https://www.eso.org/sci/activities/garching/projects/ goods/MasterSpectroscopy.html

4 http://member.ipmu.jp/john.silverman/CDFS\_vlt.html

## 2.4 Emission line galaxy selection

Star-forming galaxies at the same redshifts as our target SMGs ( 𝑧 ∼ 2 295 . ± 0 023 for ALESS 75.2 and ALESS 102.1; . 𝑧 ∼ 3 324 . ± 0 030 . for ALESS 5.1) have emission lines that are redshifted into the narrow wavelength coverage of the Br 𝛾 filter. Since the Br 𝛾 filter is near the centre of the 𝐾 𝑠 transmission a galaxy without line emission at these wavelengths will have a ( 𝐾 𝑠 -Br 𝛾 ) colour of zero. However, due to the narrow width of the Br 𝛾 filter relative to the 𝐾 𝑠 filter, galaxies with redshifts that place an emission line in the narrow Br 𝛾 filter will have a ( 𝐾 𝑠 -Br 𝛾 ) colour that is significantly greater than zero. We employ the same methodology as previous narrowband surveys (e.g. Moorwood et al. 2000; Geach et al. 2008; Sobral et al. 2013) to identify line-emitting galaxies. This methodology uses two parameters to select sources with a significant, physically-driven narrowband excess, as opposed to an excess due to random noise.

The first of these parameters, Σ , quantifies the significance of the narrowband excess compared to the expected random scatter for a source with zero ( 𝐾 𝑠 -Br 𝛾 ) colour (Bunker et al. 1995). Σ is given by:

$$\Sigma = \frac { 1 - 1 0 ^ { - 0. 4 ( B B - N B ) } } { 1 0 ^ { - 0. 4 ( Z P - N B ) } \sqrt { r m s _ { N B } ^ { 2 } + r m s _ { B B } ^ { 2 } } } \,, \quad \quad ( 1 ) \quad \text{Narrow} \\ \text{galaxi}$$

where NB and BB are the apparent magnitudes in the narrowband (Br 𝛾 ) and broadband ( 𝐾 𝑠 ) filters, respectively; ZP is the zeropoint magnitude of the narrowband images; rms NB and rms BB are the rms counts in 1 . ′′ 25 apertures for the individual narrowband and broadband quadrants, respectively. We require candidate line-emitters have Σ &gt; 3, which is consistent with previous narrowband studies (e.g. Bunker et al. 1995; Sobral et al. 2013); see Figure 1. Note that this Σ does not correspond directly to signal-to-noise (SNR) in the Br 𝛾 filter, but is a separate quantity based on counts; Σ &gt; 3 implicitly excludes sources with SNR ≲ 8 in Br 𝛾 (for details see e.g. Sobral et al. 2009).

In addition to having Σ &gt; 3 line emitters are required to have an observed equivalent width (EW) &gt; 50Å. The EW is calculated for each source using:

$$E W & = \Delta \lambda _ { B r \gamma } \frac { f _ { B r \gamma } - f _ { K _ { s } } } { f _ { K _ { s } } - f _ { B r \gamma } ( \Delta \lambda _ { B r \gamma } / \Delta \lambda _ { K _ { s } } ) } \,, & ( 2 ) \quad \text{and} \, S \, \text{$1$ imag} \\.$$

where Δ 𝜆 Br 𝛾 and Δ 𝜆 𝐾 𝑠 are the widths of the two filters and 𝑓 Br 𝛾 and 𝑓 𝐾 𝑠 are the flux densities of the source in each filter. The 50 Å lower limit on EW for a source to be selected as a line emitter was chosen to lie above the 3 𝜎 scatter in ( 𝐾 𝑠 -Br 𝛾 ) colours for bright (Br 𝛾 &gt; 19 5 . mag) sources in both pointings (Figure 1).

Before applying the selection criteria, we first account for sources that are detected with ≥ 3 𝜎 significance in the Br 𝛾 filter but &lt; 𝜎 3 in 𝐾 𝑠 . We classify these sources as non-detections in 𝐾 𝑠 , and replace their aperture magnitudes with the relevant 3 𝜎 limiting magnitude (see Table 2). However, several of these non-detections have counterparts in the S14 catalogue (within a 1 ′′ matching radius) and thus have 𝐾 𝑠 magnitudes from either TENIS (Hsieh et al. 2012), archival HAWK-I observations (Zibetti, priv. comm.), or MUSYC (Taylor et al. 2009). For these sources, we replace our HAWK-I 𝐾 𝑠 photometry with values from one of these surveys, preferentially using TENIS photometry as it is the deepest of the three (with a limiting 3 𝜎 magnitude of 𝑚 3 𝜎 lim = 24 45 mag); if no TENIS photometry is available . then we opt for the archival HAWK-I values ( 𝑚 3 𝜎 lim = 24 36 mag), . using MUSYC ( 𝑚 3 𝜎 lim = 22 55 mag) only when no photometry exists . for either of the other two. Note that while MUSYC 𝐾 𝑠 observations are the shallowest of all the data considered here (including our own), there are 11 sources for which only MUSYC photometry is available. However, all of these sources reside in regions of Pointing 102 that are (a) outside of the coverage of the TENIS and archival HAWK-I observations, and (b) close to the quadrant edges in our HAWK-I observations where the noise is at its greatest.

Using the Σ &gt; 3 and EW &gt; 50 Å selection criteria, 81 and 80 candidate line emitters are identified in Pointing 5+75 and Pointing 102, respectively (Figure 1). Of these candidates, 30 are 𝐾 𝑠 non-detections with no 𝐾 𝑠 photometry in the S14 catalogue, and thus Σ and EW are calculated by assuming that their 𝐾 𝑠 magnitudes are equal to the 3 𝜎 limiting magnitudes of our data. Since this can only provide a lower limit for the ( 𝐾 𝑠 -Br 𝛾 ) colour and thereby underestimate Σ for these sources, we do include these sources in our sample of candidate line emitters.

Finally, we visually inspect all 161 candidate line emitters, removing stars/quasi-stellar objects and image artefacts. The final sample consists of 79 and 68 candidate line emitters in Pointing 5+75 and Pointing 102, respectively (147 sources in total).

## 2.5 Identifying line emitters associated with the SMGs

Narrowband excess alone is not sufficient to identify star-forming galaxies in the same environments as our target SMGs; such an excess could be caused by a number of possible emission lines at different redshifts (see Figure 3). We therefore use the available multi-band photometric data covering our pointings to estimate photometric redshifts for the narrowband emitters in our sample. The S14 catalogue contains photometric redshift estimates for sources across the ECDFS, however after cross-matching with our data (using a matching radius of 1 ′′ ), a significant fraction ( &gt; 30 percent) of the line emitters identified in §2.4 do not have broadband counterparts in this catalogue and thus lack any redshift information. We therefore perform our own spectral energy distribution (SED) fitting using eazy-py.

To maximise the number of sources for which we can derive photometric redshifts, we extract fixed-aperture photometry at their HAWK-I Br 𝛾 positions in the UV-to-MIR images described in §2.3. Each image is astrometrically calibrated using scamp (Bertin 2006) and SWarp (Bertin 2010) to match the astrometry of our HAWKI images, and then photometrically recalibrated so that all images have a zeropoint magnitude of 30.0 mag. Photometry is extracted in fixed apertures using the photutils Python package (Bradley et al. 2022); apertures of diameter 2 . ′′ 0 are used for all images except those from Spitzer /IRAC, for which we use apertures of diameter 3 . ′′ 8 due to the larger point spread function (PSF). Aperture corrections are determined for each filter by measuring the median difference between the magnitudes measured in these apertures and those measured in adaptively scaled apertures with SExtractor for bright point sources. Final corrections are applied to each filter to account for Galactic attenuation, using values from Cardamone et al. (2010) and Hsieh et al. (2012).

eazy-py operates using a 𝜒 2 -minimisation procedure in which linear superpositions of template SEDs are tested at different redshifts to find an optimal fit to the observed fluxes (Brammer et al. 2008). In keeping with other recent studies which implement eazypy (e.g. Stevans et al. 2021; Finkelstein et al. 2022), we use the 'tweak\_fsps\_QSF\_12\_v3' set of 12 template SEDs, which cover a wide range of galaxy types and utilise a Chabrier (2003) initial mass function (IMF) and a Kriek &amp; Conroy (2013) dust attenuation law while assuming solar metallicity. An advantage of these templates is that they include emission lines, such that a narrowband excess can provide a relatively tight constraint on the redshift.

Figure 1. Colour-magnitude diagrams demonstrating the criteria described in §2.4 for the selection of candidate narrowband emitters in the HAWK-I pointings containing ALESS 5.1 &amp; 75.2 (left), and ALESS 102.1 (right). All sources detected in the Br 𝛾 observations are shown and sources identified as narrowband emitters are highlighted. Open symbols represent candidate narrowband emitters which were removed from the sample following visual checks. Sources that are detected in Br 𝛾 but are undetected in our 𝐾 𝑠 data and have no 𝐾 𝑠 photometry in S14, are shown as lower limits. The Σ = 3 curve for the average properties and the observed-frame equivalent width cut for each field are shown. The solid horizontal line marks a 𝐾 𝑠 -Br 𝛾 colour of zero. The target SMGs and other ALESS SMGs with counterparts in our HAWK-I data are highlighted. Two of the target SMGs (ALESS 5.1 and 75.2) and several other ALESS SMGs in these fields are not detected and are therefore not shown. The black cross in the bottom left corner of each panel shows the mean uncertainties in colour and Br 𝛾 magnitude.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00063v1-figure-b645188d-ceaa-444d-8dd3-6a376d164844.jpg)

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00063v1-figure-caeff4a0-977a-4ad7-9c42-b75986ede9f7.jpg)

As discussed in §2.3, there have been several spectroscopic studies in the ECDFS, from which spectroscopic redshifts have been obtained for a number of galaxies across the field. Using a matching radius of 1 . ′′ 5, we cross-match our data with catalogues from VVDS (Le Fèvre et al. 2005), the GOODS/VIMOS survey (Popesso et al. 2009), the ECDFS spectroscopic survey (Silverman et al. 2010), and the spectroscopic studies conducted by Treister et al. (2009) and Danielson et al. (2017). This gives spectroscopic redshifts for 163 ( ∼ 4 1 percent) of the 3929 sources detected in our HAWK-I imaging, . including nine for which photometric redshifts could not be estimated due to insufficient photometry. Seven of the 163 sources with spectroscopic redshifts are emission line galaxies selected in §2.4; the spectroscopic redshifts for these sources are used for our analyses. We compare the photometric and spectroscopic redshifts for our HAWK-I sources in Figure 2. There is strong agreement between the photometric and spectroscopic redshifts, which is quantified using the normalised median absolute deviation (NMAD) of Δ 𝑧 :

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00063v1-figure-05bc9170-ca68-4d3c-826b-08f1a9cb6ccb.jpg)

$$\sigma _ { N M A D } = 1. 4 8 \times \text{median} \left ( \left | \frac { \Delta z - \text{median} ( \Delta z ) } { 1 + z _ { \text{spec} } } \right | \right ) \,, \quad \quad \quad ( 3 ) \quad \text{spectro} \colon$$

where 𝑧 spec is the spectroscopic redshift and Δ 𝑧 = 𝑧 spec -𝑧 phot . We obtain 𝜎 NMAD = 0 062 when considering all 152 HAWK-I detec-. tions with photometric and spectroscopic redshifts.

Only five ( ∼ 3 4 percent) of our 147 emission line galaxies have . neither spectroscopic nor photometric redshifts, the latter being due to a lack of photometry with sufficient depth. Figure 3 shows the redshifts of the remaining 142 emission line galaxies, compared with the significance of their narrowband excess ( Σ ; Equation 1). Peaks in the redshift distribution are visible at 𝑧 ∼ 2 3 (both pointings) and . 𝑧 ∼ 3 3 (Pointing 5+75 only), as expected of H . 𝛼 and [Oiii] in the environments of the target SMGs. We select as H 𝛼 ([Oiii]) emitters any galaxies for which 2 23 . &lt; 𝑧 &lt; 2 37 (3 23 . . &lt; 𝑧 &lt; 3 41), where .

Figure 2. Photometric redshifts derived using eazy-py compared to spectroscopic redshifts for all sources detected in HAWK-I Br 𝛾 with archival spectroscopic redshifts (from Le Fèvre et al. 2005; Popesso et al. 2009; Treister et al. 2009; Balestra et al. 2010; Silverman et al. 2010; Danielson et al. 2017). Galaxies included in our final sample of candidate line emitters (see 2.4) are highlighted. The redshifts at which common extragalactic emission lines enter the Br 𝛾 filter are shown using horizontal and vertical lines; dashed lines highlight H 𝛼 and [Oiii], which the emission lines of interest in this study. The diagonal line shows a one-to-one correspondence; the scatter is low and majority of sources have photometric redshifts that are consistent with their spectroscopic redshifts.

these redshift ranges correspond to 3 × the FWHM of the Br 𝛾 filter when H 𝛼 ([Oiii]) has redshifted to the centre. We represent these selection criteria with shaded regions in Figure 3; the highlighted galaxies are henceforth assumed to be H 𝛼 and [Oiii] emitters at

Table 3. Summary of the sample at each stage of the selection process described in §2.4 and §2.5.

|                                     | Number per pointing   | Number per pointing   | Total   |
|-------------------------------------|-----------------------|-----------------------|---------|
|                                     | Pointing 5+75         | Pointing 102          |         |
| Br 𝛾 detections                     | 2175                  | 1754                  | 3929    |
| Line emitter candidates (initial)   | 81                    | 80                    | 161     |
| Line emitter candidates (confirmed) | 79                    | 68                    | 147     |
| H 𝛼 candidates                      | 44                    | 11                    | 55      |
| [Oiii] candidates                   | 4                     | 2                     | 6       |

similar redshifts as the target SMGs. We identify 44 H 𝛼 emitters and 4 [Oiii] emitters in Pointing 5+75, and in Pointing 102 there are 11 H 𝛼 emitters ([Oiii] emitters in Pointing 102 are not further considered because there is no SMG at 𝑧 ∼ 3 3 in this pointing). . Table 3 summarises the results of each step in the sample selection. Wenote that all of these H 𝛼 and [Oiii] candidates have an SNR &gt; 8 5 . in the Br 𝛾 filter as a natural consequence of our selection process (see also §2.4). We therefore do not expect the sizes of these samples to be significantly affected by Eddington bias (Eddington 1913).

To identify any AGN in the sample we use a 1 ′′ matching radius to locate counterparts in the Lehmer et al. (2005) Chandra point source catalogue. None of the [Oiii] emitters and only one of the H 𝛼 emitters (2.3%) is an X-ray luminous AGN, which is consistent with the rate of X-ray AGN in blank-field surveys of H 𝛼 emitters at the same redshift (e.g. 1 8 . ± 1 3% in Calhau et al. 2017). Since the AGN . fraction is the same as in field surveys then this galaxy is kept in our sample to enable a fair like-for-like comparison between the SMG fields and blank field H 𝛼 emitters.

Figure 4 shows the distributions of emission line galaxies across the HAWK-I pointings, with H 𝛼 and [Oiii] emitters highlighted. For all three target SMGs, the companion galaxies are spread across the entire field of view and therefore span several physical Mpc. This is consistent with expectations from simulations (e.g. Chiang et al. 2013; Muldrew et al. 2015; Yajima et al. 2022, see also §3.3), in which protoclusters are seen to extend over several Mpc, such that the entire structure is unlikely to be captured by a single HAWK-I pointing. We also note the presence of a dense clump of seven H 𝛼 emitters (three of which are spectroscopically confirmed at 𝑧 ∼ 2 3) in the northeast . of Pointing 5+75, which coincides with a photometrically-identified Ly 𝛼 blob at 𝑧 ∼ 2 3 (CDFS-LAB03; Yang et al. 2010). This system . will be discussed further in §3.3.

Of the three SMGs targeted, only ALESS 102.1 is identified as an H 𝛼 (or [Oiii]) emitter in our data. Danielson et al. (2017) did not identify ALESS 102.1 as an H 𝛼 emitter in their spectroscopy, because the wavelength coverage with VLT/FORS2 and VLT/VIMOS does not cover H 𝛼 at 𝑧 ∼ 2 3. . The original spectroscopic redshift for ALESS5.1 is from CO(4-3) (Birkin et al. 2021) and no emission lines were observed in Keck/DEIMOS, Keck/MOSFIRE or VLT/XSHOOTER observations (Danielson et al. 2017); this is likely because the redshifted [Oiii] line clashes with a bright OH -sky line for this source (Ramsay et al. 1992). For ALESS 75.2 the original spectroscopic redshift was measured, in part, thanks to a faint H 𝛼 line detected in Keck/MOSFIRE observations (Danielson et al. 2017), which is below the detection limit of our data.

## 3 RESULTS, ANALYSIS &amp; DISCUSSION

## 3.1 Measuring luminosity functions

In order to quantify whether the SMGs reside in significant overdensities of H 𝛼 or [Oiii] emitters, a comparison to the blank field needs to be drawn. The High Redshift ( 𝑧 ) Emission Line Survey (HiZELS; Geach et al. 2008) is a large narrowband survey of emission line galaxies, including H 𝛼 emitters at 𝑧 = 2 23 (Sobral et al. 2013) and . [Oiii] emitters at 𝑧 = 3 24 (Khostovan et al. 2015) in COSMOS. The . HiZELSresults are therefore representative of H 𝛼 and [Oiii] emitters in regions of average density at redshifts similar to those of our target SMGs, and we use their luminosity functions as a blank-field sample for comparison with our results.

To construct luminosity functions for our H 𝛼 and [Oiii] emitters we bin them according to line luminosities, making corrections to the observed number counts in each bin to account for completeness, contamination from other emission lines, dust attenuation, and the shape of the narrowband filter profile. Each of these steps is described in more detail below.

## 3.1.1 Survey volumes

Approximating the Br 𝛾 filter profile as a top-hat function with width equal to the FWHM of the filter ( Δ 𝜆 = 0 030 . 𝜇 m), the H 𝛼 emission line should be detectable from 𝑧 min = 2 276 to . 𝑧 max = 2 322, cor-. responding to a comoving volume per square degree of 5 86 . × 10 5 cMpc 3 deg -2 . The redshift range within which [Oiii] can be detected extends from 𝑧 min = 3 294 to . 𝑧 max = 3 354, which gives . 1 04 . × 10 6 cMpc 3 deg -2 . Accounting for the high-noise regions that were masked prior to source detection (see §2.2) and the gaps between the HAWK-I detector chips, the surveyed areas in Pointing 5+75 and Pointing 102 are 0.0117 and 0.0118 deg 2 , respectively. The volumes probed are therefore: 6859 cMpc 3 for H 𝛼 in Pointing 5+75; 6891 cMpc 3 for H 𝛼 in Pointing 102; 12180 cMpc 3 for [Oiii] in Pointing 5+75; 12230 cMpc 3 for [Oiii] in Pointing 102. In §3.1.6 we correct the derived luminosity functions to account for the fact that the Br 𝛾 filter is not a perfect top-hat function, which leads to the volume probed being slightly different for sources with different luminosities.

## 3.1.2 Completeness correction

It is possible that real galaxies with weak emission lines were missed in our selection process (§2.4) despite actually meeting the selection criteria: the sample is incomplete at low emission line fluxes. We correct for this using the method employed by Sobral et al. (2013), applying it separately for each quadrant of each pointing due to the variation in depth between detector chips (see Table 2). For each emission line (H 𝛼 and [Oiii]), we select sources that failed to meet the emission line galaxy selection criteria (i.e. sources for which EW &lt; 50 Å and/or Σ &lt; 3) with redshifts within the range used to identify the targeted emission line (see §2.5). Due to the size of these samples, we generate ∼ 1000 mock galaxies by randomly varying the 𝐾 𝑠 and Br 𝛾 magnitudes of the selected galaxies according to their uncertainties (assuming a Gaussian probability distribution for each magnitude) and placing them at random positions within their quadrant, removing any sources for which these changes result in them being classed as a line emitter. We then artificially inject line flux to each galaxy in this bolstered sample of non-line emitters, beginning with 10 -22 erg s -1 cm -2 and incrementally increasing it

z

Figure 3. The distributions of redshifts for the emission line galaxies in Pointing 5+75 (left) and Pointing 102 (right) compared with their emission-line significance, Σ . Photometric redshifts are computed using eazy-py, with archival spectroscopic redshifts included where available (§2.5). H 𝛼 and [Oiii] emitters are highlighted, and shaded regions show the redshift ranges used to select them. Peaks in the redshift distributions at these redshifts may be driven by overdensities of these line emitters. Horizontal dashed lines show the redshifts at which other common extragalactic emission lines enter the Br 𝛾 filter.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00063v1-figure-4dc45708-a1a9-4b19-b1ac-00963c9bc6e2.jpg)

Figure 4. Spatial distribution of the emission line galaxies and the other Br 𝛾 detections in Pointing 5+75 (left) and Pointing 102 (right). Also shown are the positions of the target SMGs and other ALESS SMGs in these areas, although the redshifts of the non-target SMGs are either unknown or outside the ranges that would place the H 𝛼 or [Oiii] emission lines in the Br 𝛾 filter (Danielson et al. 2017; Birkin et al. 2021). H 𝛼 and [Oiii] candidates are indicated. While Pointing 102 does contain [Oiii] candidates, they are not shown here because there are no ALESS SMGs at 𝑧 ∼ 3 3 in this pointing. For all three SMGs, . the candidate companion galaxies are distributed across the entire HAWK-I field of view, corresponding to physical spans of a few Mpc, as expected from protocluster simulations (e.g. Chiang et al. 2013; Muldrew et al. 2015; Yajima et al. 2022). Dashed circles show the boundaries of annuli used to measure radial trends in the density of companion galaxies (§3.3).

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00063v1-figure-323c553e-1d9e-4f0d-9dc6-351966b3b051.jpg)

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00063v1-figure-0724f431-feed-478f-b35d-b82ab85242e0.jpg)

by 0.05 dex. Line fluxes are calculated as

$$F _ { \text{line} } & = \Delta \lambda _ { \text{Br} \gamma } \frac { f _ { \text{Br} \gamma } - f _ { K _ { s } } } { 1 - ( \Delta \lambda _ { \text{Br} \gamma } / \Delta \lambda _ { K _ { s } } ) } \,, & \quad \text{line tiu} \\ & & \text{interatic}$$

luminosity functions. The uncertainty in the completeness at a given line flux is estimated by regenerating the mock galaxies 1000 times and measuring the standard deviation in the completeness across all iterations.

where 𝑓 Br 𝛾 and 𝑓 𝐾 𝑠 are the Br 𝛾 and 𝐾 𝑠 flux densities, respectively, in units of erg s -1 cm -2 Å 1 . With each increment of injected line flux, -we recalculate the EW and Σ and reapply the line emitter selection criteria to determine the catalogue completeness as a function of line flux. This is used to estimate the completeness corrections for our

## 3.1.3 Removing [N ii ] contamination

The H 𝛼 emission line lies in between a doublet of [Nii] lines at rest-frame wavelengths of 6548 Å and 6583 Å, which will contribute

to the measured Br 𝛾 flux density and therefore affect the observed EW and emission line flux. Using spectroscopic data taken with Subaru/FMOS and Keck/MOSFIRE, Sobral et al. (2015) observed an anti-correlation between the [Nii] 𝜆 6583-to-H 𝛼 line flux ratio and the rest-frame EW (EWrest = EW /( 1 + 𝑧 ) ) for the H 𝛼 emitters in HiZELS, deriving the following empirical relation:

$$\frac { F _ { [ N _ { m } ] } } { F _ { H \alpha } } = - 0. 2 9 6 \times \log _ { 1 0 } ( E W _ { \text{rest}, H \alpha + N _ { m } } ) + 0. 8 \,. \quad \quad ( 5 ) \quad \text{ are cal} \\ \text{ and with }.$$

Weadopt this relation to apply corrections to the line fluxes of all H 𝛼 emitters in our sample, resulting in a median decrease of 11 + 9 -5 percent in emission line flux.

## 3.1.4 Relative contributions from [O iii ] 𝜆 5007 , [O iii ] 𝜆 4959 and H 𝛽

Thus far only the [Oiii] 𝜆 5007 emisson line has been considered in the discussion of [Oiii] emitters at 𝑧 ∼ 3 3. However, this line is part of . a doublet, with its counterpart residing at a rest-frame wavelength of 4959 Å, and there is a narrow range of redshifts ( 𝑧 = 3 336-3 344) in . . which both lines can contribute to a galaxy's Br 𝛾 flux. Furthermore, while the H 𝛽 emission line is sufficiently separated from the [Oiii] doublet to avoid contaminating the measured [Oiii] line flux, it is still close enough such that there is the potential for H 𝛽 emitters to be misidentified as [Oiii] emitters (see Figure 3). Therefore, rather than try and separate our sample into [Oiii] 𝜆 5007, [Oiii] 𝜆 4959 and H 𝛽 emitters, we present a combined [Oiii]+H 𝛽 luminosity function; this also allows for a consistent comparison with the blank-field [Oiii]+H 𝛽 luminosity function from Khostovan et al. (2015).

Wedo however take into account the results of Sobral et al. (2015) when estimating the total volumes probed by the Br 𝛾 filter in the search for [Oiii] emitters: using spectroscopy, Sobral et al. (2015) find that for HiZELS ∼ 50 percent of photometrically-selected [Oiii]+H 𝛽 emitters at 𝑧 ∼ 1 4 are [Oiii] . 𝜆 5007, ∼ 27 percent are [Oiii] 𝜆 4959, ∼ 16 percent are H 𝛽 , with the remaining ∼ 7 percent being simultaneous detections of [Oiii] 𝜆 5007 and [Oiii] 𝜆 4959. Based on these results, Sobral et al. (2015) then add to the total volume probed (i.e. the volume probed if searching for [Oiii] 𝜆 5007 emitters) 16 percent of the volume that would be probed had their search been for H 𝛽 , and 25 percent of the volume had they been searching for [Oiii] 𝜆 4959. We thus apply similar corrections to our total volume probed for [Oiii] 𝜆 5007 emitters.

## 3.1.5 Corrections for dust attenuation

Dust in star-forming galaxies reprocesses light emitted in the restframe UV and optical, and can therefore reduce the amount of H 𝛼 and [Oiii] flux observed. In order to estimate the intrinsic brightness of the emission lines (i.e. their integrated luminosities), one has to correct for the effect of dust attenuation. For an attenuation of 𝐴 line (mag) at the emission line wavelength, the conversion from line flux to intrinsic line luminosity is

$$L _ { \text{line} } & = 4 \pi D _ { L } ^ { 2 } F _ { \text{line} } \times 1 0 ^ { 0. 4 A _ { \text{line} } } \,, & ( 6 ) & \quad \text{Due}$$

where 𝐷 L is the luminosity distance. We follow Sobral et al. (2013) and assume an attenuation at the H 𝛼 wavelength of 𝐴 H 𝛼 = 1 mag, which is based on previous HiZELS studies (Garn et al. 2010; Sobral et al. 2012). Khostovan et al. (2015) do not correct for dust attenuation when plotting their luminosity functions, so we also leave our [Oiii]+H 𝛽 luminosities uncorrected to ensure a consistent comparison. Khostovan et al. (2015) later go on to calculate the dustcorrected star formation rates (SFRs) of their galaxies, where they then assume an attenuation of 𝐴 [Oiii]+H 𝛽 = 1 35 mag, derived by .

assuming 𝐴 H 𝛼 = 1 mag and using a Calzetti et al. (2000) dust attenuation curve. We thus adopt the same correction when calculating our own SFRs (see §3.4).

## 3.1.6 Filter profile volume corrections

The comoving volumes used for our luminosity functions (§3.1.1) are calculated by approximating the Br 𝛾 filter as a top-hat filter with width equal to the Br 𝛾 FWHM. Since the filter profile is not a tophat in reality, this introduces two main effects which need to be accounted for when estimating the galaxy number densities. Firstly, bright emitters whose line falls near the edges of the Br 𝛾 filter will suffer a significant loss of line flux and thus appear to be fainter than they really are. This produces an overall bias towards faint sources in our sample. Secondly, any faint emitters close to the filter edges might be missed from our sample, and are therefore only detectable over a narrower redshift range (and thus a smaller volume) than their bright counterparts.

To correct for these effects, we follow the method used by Sobral et al. (2013) and Khostovan et al. (2015), as first proposed in Sobral et al. (2009). An initial Schechter fit is performed to the uncorrected 5 data. We then generate a mock sample of 10 5 fake sources with a luminosity distribution that is weighted by the uncorrected Schechter function. These sources are randomly assigned redshifts with a uniform distribution across the whole possible Br 𝛾 coverage. They are then convolved through the Br 𝛾 filter profile such that their luminosities decrease according to their assigned redshift (i.e. according to the position of the redshifted emission line in the filter profile) and rebinned using the same luminosity bins as for the uncorrected data. Comparing the resultant distribution to the input distribution reveals that bright sources are underestimated relative to the fainter sources, as expected. The real data are corrected using the ratio of these distributions.

## 3.1.7 Fitting Schechter functions

Finally, we perform fits to the corrected data using a Schechter function (Schechter 1976):

$$\dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \ delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta\delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \dots \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \ delta \ delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \dia \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \Delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta \delta$$

where Φ ( 𝐿 ) is the number density at luminosity 𝐿 , Φ ∗ is the normalisation of the luminosity function, 𝐿 ∗ is the characteristic luminosity, and 𝛼 is the slope at the faint end of the luminosity function, where the power-law component dominates.

The faintest bins (open symbols in Figure 5) are excluded from each fit due to their low completeness. For the H 𝛼 emitters, we take 'low completeness' to mean that the low-luminosity edge of the bin lies below the 30 percent completeness limit. The line luminosities of all [Oiii]+H 𝛽 emitters lie above the 90 percent completeness threshold; we therefore do not exclude any from the fit.

Duetothesmallnumberofbinsleft available for fitting, it is impossible to reliably constrain all three free parameters of the Schechter function simultaneously. For the H 𝛼 luminosity functions, we therefore fix the faint-end slope, 𝛼 , to the value of -1 59 obtained by . Sobral et al. (2013) at 𝑧 = 2 23. For the [Oiii]+H . 𝛽 emitters, we only have one available bin and thus fix both 𝛼 and log ( 𝐿 ∗ / erg s -1 ) to the

5 'Uncorrected' only in terms of the filter profile correction; the results have already been corrected for line completeness and dust attenuation by this stage.

values for the 𝑧 = 3 24 sample of [Oiii]+H . 𝛽 from Khostovan et al. (2015), which are -1 60 and 42.83, respectively. Thus our [Oiii]+H . 𝛽 luminosity function is effectively a renormalised version of the Khostovan et al. (2015) result, with Φ ∗ being the only free parameter. In addition to the H 𝛼 luminosity functions of the individual pointings, we also construct fits to the combined sample of H 𝛼 emitters from both SMG fields, as this provides a more general view of SMGs at 𝑧 ∼ 2 3 with improved statistics. .

The best-fit parameters for each luminosity function are summarised in Table 4. Uncertainties are estimated for each free parameter by randomly perturbing the bin heights according to their uncertainties and then recalculating the fit, and repeating this process until 10 5 fits have been made. The 1 𝜎 confidence interval for each parameter is then estimated using the 16th and 84th percentiles of the best-fit values.

## 3.2 Analysing luminosity functions

We next use the luminosity functions to assess whether the targeted SMGs reside in overdensities of emission-line galaxies. Figure 5 compares the observed luminosity functions from the SMG fields to those from the blank field surveys of Sobral et al. (2013) and Khostovan et al. (2015). The environment of ALESS 75.2 shows signs of being overdense relative to the field at 𝑧 ∼ 2 3, with most bins lying . significantly above the blank field luminosity function. Conversely the environments of ALESS 5.1 and 102.1 are broadly consistent with the blank field luminosity functions at their respective epochs. An overdensity remains when the H 𝛼 emitters from both fields at 𝑧 ∼ 2 3 are combined, implying that on average SMGs at this epoch . reside in overdense, protocluster-like environments, which qualitatively consistent with clustering results (e.g. Hickox et al. 2012; Wilkinson et al. 2017; Stach et al. 2021). The contrast between the individual H 𝛼 luminosity functions suggests that there is significant variation across SMG environments, although observations of additional SMGs are required to confirm and quantify the field-to-field variation. Furthermore, as explored later in this section and shown in Figure 4, the field around ALESS 102.1 is itself overdense on smaller scales.

To quantitatively compare the SMG field and blank field luminosity functions we consider the parameters of the Schechter function fits (§ 3.1.7). The parameters of Schechter function fitting are often correlated, so in Figure 6 we show the uncertainties of the luminosity function parameters in the 𝐿 ∗ -Φ ∗ plane (as described in §3.1.7 the faint-end slope, 𝛼 , is fixed), comparing our SMG fields with the blank fields at similar redshifts. For the individual SMG environments at 𝑧 ∼ 2 3 the fit parameters are offset from those of the blank field, . although for the ALESS 102.1 region the offset is only at the ∼ 1 𝜎 level. These separations are driven by a higher 𝐿 ∗ , and, in the case of ALESS 75.2, by a larger Φ ∗ , which implies that this environment is preferentially overdense in bright line emitters compared to the blank field. Meanwhile, the environment of ALESS 5.1 exhibits an offset of &lt; 𝜎 1 relative to the blank-field value of Φ ∗ at 𝑧 ∼ 3 3, implying . this SMG does not reside in an overdnensity.

We quantify the galaxy overdensity in each sample of H 𝛼 and [Oiii]+H 𝛽 emitters in two ways. Firstly, we calculate the ratio of the Φ ∗ from the best fit Schechter function to those from the relevant blank-field luminosity functions, Φ ∗ / Φ ∗ field . This ratio tells us how much higher the 'knee' of each SMG-field luminosity function is relative to the blank-field. These ratios are presented in Table 4. The value for the [Oiii]+H 𝛽 luminosity function suggests that the environment of ALESS 5.1 at 𝑧 ∼ 3 3 is 1 2 . . + 0 6 . -0 4 . times as dense as the field, i.e. it is consistent with the blank field. For the H 𝛼 emitters at 𝑧 ∼ 2 3, . 𝐿 ∗ is also a free parameter in the Schechter fits, and the 𝐿 ∗ -Φ ∗ correlation means that we must first refit the data with 𝐿 ∗ fixed to the 𝑧 ∼ 2 3 blank-field value from Sobral et al. (2013), . i.e. log ( 𝐿 ∗ / erg s -1 ) = 42 83. These fits give . Φ ∗ / Φ ∗ field values of 3 6 . + 0 6 . -0 6 . and 1 7 . + 0 3 . -0 3 . for the ALESS 75.2 and ALESS 102.1 fields, respectively. The combined sample of H 𝛼 emitters from both SMG environments suggests that the average SMG environment at 𝑧 ∼ 2 3 . is 2 6 . + 0 4 . -0 4 . times more dense than the blank field at this epoch.

To derive a more representative estimate of the galaxy overdensity in each environment, we also estimate the number of H 𝛼 or [Oiii]+H 𝛽 emitters that one would expect to find in a blank field with the volumes probed by our observations. To do this, we integrate the field luminosity functions across the luminosity range covered by our data, excluding low completeness bins; i.e. we integrate across the ranges 42 5 . ≤ log ( 𝐿 ∗ H 𝛼 / erg s -1 ) &lt; 43 8 and . 42 4 . ≤ log ( 𝐿 ∗ [Oiii]+H 𝛽 / erg s -1 ) &lt; 42 7 and multiply by the volumes . probed in each HAWK-I pointing to estimate the expected number counts in an equivalent blank field, 𝑁 field . Since these field galaxies would have contributed to the observed number counts, we quantify the galaxy overdensity in each environment using

$$\begin{smallmatrix} \dots \dots \\ \text{Kbos-} \\ \text{gns of} \end{smallmatrix} \quad \delta _ { g } = \frac { N _ { \text{total} } - N _ { \text{field} } } { N _ { \text{field} } }$$

where 𝑁 total is the sum of the counts in our complete bins. Uncertainties in 𝑁 total are determined by adding in quadrature the uncertainties in the bin counts. For 𝑁 field the uncertainties are estimated by randomly permuting the blank-field Schechter parameters within their uncertainties prior to integrating, then repeating the process 10 5 times and using the 16th and 84th percentiles of the resultant number counts to define the 1 𝜎 confidence interval.

The values of 𝛿 𝑔 for each sample of H 𝛼 and [Oiii]+H 𝛽 emitters are summarised in Table 4 along with the significance of this overdensity, 𝜎 𝛿 . Based on these values, the environments of ALESS 5.1, 75.2, and 102.1 are overdense by factors of 0 2 . + 2 5 . -0 7 . ( 0 3 . 𝜎 𝛿 ) , 2 6 . + 1 4 . -1 2 . ( 2 3 . 𝜎 𝛿 ) , and 0 2 . + 0 6 . -0 5 . ( 0 5 . 𝜎 𝛿 ) , respectively. If the samples of H 𝛼 emitters from both pointings are considered as one, then the SMG environments at 𝑧 ∼ 2 3 are overdense by a factor of 1 5 . . + 1 0 . -0 8 . ( 1 9 . 𝜎 𝛿 ) on average. The above uncertainties do not account for cosmic variance, which could cause a factor ∼ 2 difference in number counts, as based on the H 𝛼 emitters in two equal depth HAWK-I pointings in COSMOS and UDS (Sobral et al. 2013). Including cosmic variance in our calculations does not change our overall conclusions that the ALESS 75.2 and the combined 𝑧 ∼ 2 3 SMG fields are overdense, nor does it . affect the conclusions that ALESS 5.1 and ALESS 102.1 reside in environments consistent with the blank field.

While Pointing 102 as a whole is not overdense, the majority of the H 𝛼 emitters in this pointing are contained within the same quadrant as the SMG, as can be seen in Figure 4 (see also §3.3). We therefore recalculate 𝛿 𝑔 for this SMG environment, this time considering only the volume probed within that quadrant (1722 cMpc 3 ), finding 𝛿 𝑔 = 3 8 . + 2 4 . -1 8 . ( 2 1 . 𝜎 ) in this area, which suggests that ALESS 102.1 actually does reside in an overdense environment with a physical scale of ∼ 1 6 Mpc. This high concentration of galaxies surrounding the SMG . could be indicative of structure formation on smaller scales than those of protoclusters and it is possible that ALESS 102.1 resides in a protogroup (e.g. Diener et al. 2013). We discuss the spatial distribution of companion galaxies in each SMG environment in more detail in §3.3.

The question remains as to whether the target SMGs reside in protoclusters, which will evolve into bound clusters by the present day. To learn more, we compare the overdensities in the SMG fields

Figure 5. Luminosity functions of H 𝛼 (left) and [Oiii] (right) emitters identified around SMGs at 𝑧 ∼ 2 3 and . 𝑧 ∼ 3 3, respectively. Open symbols represent . bins that are highly incomplete and are thus excluded from the fits (see §3.1.7). The data are compared with luminosity functions from blank field studies at similar redshifts (Sobral et al. 2013; Khostovan et al. 2015), which are highlighted with coloured solid curves. Shaded regions represent the 1 𝜎 uncertainties on each luminosity function. For our H 𝛼 luminosity functions (left) the faint-end slope is fixed to the value from the corresponding blank-field luminosity function: 𝛼 = -1 59 (Sobral et al. 2013) and dashed and dotted curves show the fitted Schechter functions for the environments of ALESS 75.2 and ALESS 102.1, . respectively. The thick, solid black curve shows the Schechter function obtained by fitting to the data for our combined sample of H 𝛼 emitters at 𝑧 ∼ 2 3 (black . data points); the grey shaded region shows the 1 𝜎 confidence region for this fit. The thin black line shows another Schechter function obtained by fitting to the black data points, but with 𝐿 ∗ fixed to the blank-field value of log ( 𝐿 ∗ / erg s -1 ) = 42 87 (Sobral et al. 2013). In the right panel, the thick solid black curve . and grey shaded region shows the result of scaling up the blank-field luminosity function from Khostovan et al. (2015) to fit to the single bin of [Oiii] emitters from the environment of ALESS 5.1. Comparison with the blank-field luminosity functions reveals that ALESS 5.1, ALESS 75.2, and ALESS 102.1 reside in environments with overdensity parameters of 𝛿 𝑔 = 0 2 . + 2 5 . -0 7 . , 2 6 . + 1 4 . -1 2 . , and 0 2 . + 0 6 . -0 5 . , respectively. On average, the SMGs at 𝑧 ∼ 2 3 reside in environments with . galaxy overdensities of 𝛿 𝑔 = 1 5 . + 1 0 . -0 8 . .

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00063v1-figure-14b935a3-27c1-43f6-b325-bae8bdc77492.jpg)

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00063v1-figure-ad253887-8d12-414c-8e24-8516f9b399d8.jpg)

Table 4. Summary of the best-fit Schechter parameters for companion galaxies in the environments of the target SMGs, along with comparisons to the blank field at similar redshifts. In all cases, the faint-end slope of the luminosity function, 𝛼 , is fixed to the value from the relevant blank-field luminosity function and, where indicated, the characteristic luminosity, 𝐿 ∗ , is also fixed to the blank-field values. The values of Φ ∗ field are taken from the relevant blank-field Schechter functions. All blank-field parameters are from (Sobral et al. 2013) and (Khostovan et al. 2015).

| SMG environment   | log ( 𝐿 ∗ / erg s - 1 )   | log ( Φ ∗ / Mpc - 3 ) 𝑎    | log ( Φ ∗ fixed 𝐿 ∗ / Mpc - 3 ) 𝑏   | Φ ∗ / Φ ∗ field 𝑎        | Φ ∗ fixed 𝐿 ∗ / Φ ∗ field 𝑏   | 𝛿 𝑔 𝑐                      | 𝜎 𝛿 𝑑   |
|-------------------|---------------------------|----------------------------|-------------------------------------|--------------------------|-------------------------------|----------------------------|---------|
| ALESS 75.2        | 43 . 18 + 0 . 42 - 0 . 28 | - 2 . 57 + 0 . 29 - 0 . 39 | - 2 . 22 + 0 . 05 - 0 . 09          | 1 . 62 + 1 . 05 - 0 . 80 | 3 . 63 + 0 . 58 - 0 . 59      | 2 . 61 + 1 . 42 - 1 . 15   | 2.3     |
| ALESS 102.1       | 43 . 28 + 0 . 50 - 0 . 28 | - 3 . 05 + 0 . 17 - 0 . 42 | - 2 . 55 + 0 . 08 - 0 . 12          | 0 . 54 + 0 . 22 - 0 . 23 | 1 . 70 + 0 . 30 - 0 . 32      | 0 . 21 + 0 . 55 - 0 . 45 𝑒 | 0 . 5 𝑒 |
| SMGs at 𝑧 ∼ 2 . 3 | 43 . 00 + 0 . 12 - 0 . 10 | - 2 . 53 + 0 . 12 - 0 . 17 | - 2 . 36 + 0 . 04 - 0 . 08          | 1 . 78 + 0 . 41 - 0 . 43 | 2 . 63 + 0 . 41 - 0 . 42      | 1 . 51 + 0 . 98 - 0 . 80   | 1.9     |
| ALESS 5.1         | 42 . 83 (fixed)           | -                          | - 3 . 22 + 0 . 09 - 0 . 26          | -                        | 1 . 23 + 0 . 58 - 0 . 40      | 0 . 22 + 2 . 50 - 0 . 66   | 0.3     |

- 𝑎 Φ ∗ as measured when both Φ ∗ and 𝐿 ∗ are free parameters ( 𝛼 is always fixed to the blank-field values).
- 𝑏 Φ ∗ fixed 𝐿 ∗ is obtained by fitting a Schechter function to the data with both 𝐿 ∗ and 𝛼 fixed to the blank-field values. The values of 𝐿 ∗ are taken from the relevant blank-field Schechter functions.
- 𝑐 Galaxy overdensity, 𝛿 𝑔 = ( 𝑁 total / 𝑁 field ) -1; see §3.2.
- 𝑑 Significance of the galaxy overdensity, 𝛿 𝑔 .
- 𝑒 ALESS 102.1 has 𝛿 𝑔 = 3 8 . + 2 4 . -1 8 . (i.e. 𝜎 𝛿 = 2 1) when considering only the HAWK-I quadrant containing the SMG (§3.2). .

with previous studies of protoclusters. However, protoclusters exhibit a wide range of galaxy overdensities; a 'typical' value of 𝛿 𝑔 is not well-defined, though we highlight here structures at similar redshifts to our target SMGs. For example, 𝛿 𝑔 = 2 5 for the . 𝑧 = 1 99 protoclus-. ter in the GOODS-N field (Chapman et al. 2009). The protoclusters 4C 10.48 ( 𝑧 = 2 35) and 4C 23.56 ( . 𝑧 = 2 48), which were both iden-. tified using narrowband selection of H 𝛼 emitters around luminous radio galaxies, were found to have overdensities of 𝛿 𝑔 = 11 + 2 -2 and 𝛿 𝑔 = 4 + 5 -3 , respectively (Hatch et al. 2011; Tanaka et al. 2011). Similarly, Matsuda et al. (2011) used a narrowband search for H 𝛼 emitters at 𝑧 = 2 23 to identify overdensities of . 𝛿 𝑔 ∼ 3 2 , , and 2 around a quasi-stellar object overdensity, a high-redshift radio galaxy, and an overdensity of SMGs and optically faint radio galaxies, respectively. The protocluster Cl J0227-0421 at 𝑧 = 3 29 is overdense by a factor . of 10 5 . ± 2 8 (Lemaux et al. 2014). Two protoclusters in the COS-. MOSfield at 𝑧 = 2 10 and . 𝑧 = 2 47 were found to have overdensities . of 𝛿 𝑔 ∼ 8 and 𝛿 𝑔 ∼ 3 3, respectively (Yuan et al. 2014; Chiang et al. . 2017). Zheng et al. (2021) confirm overdensities of H 𝛼 emitters in two protocluster candidates, BOSS1244 and BOSS1542, with overdensity factors of 𝛿 𝑔 = 5 6 . ± 0 3 and . 𝛿 𝑔 = 4 9 . ± 0 3, respectively. . Comparing 𝛿 𝑔 for these protoclusters with our values, we posit that the environments of ALESS 75.2 and ALESS 102.1 (and thus of

Figure 6. Contours showing the correlated uncertainties on the Schechter parameter fits to the luminosity functions shown in Figure 5. For all the SMG fields, the faint-end slope, 𝛼 , is fixed to match to the blank-field luminosity functions from Sobral et al. (2013) and Khostovan et al. (2015). Single contours are at the 1 𝜎 level; the combined 𝑧 ∼ 2 3 data has 1, 2 and 3 . 𝜎 contours shown. For ALESS 5.1, only the 1 𝜎 error bars are shown as 𝐿 ∗ is fixed. Both of the luminosity functions for SMGs at 𝑧 ∼ 2 3 are separated from the . corresponding blank-field luminosity function in 𝐿 ∗ -Φ ∗ space, although for ALESS 102.1 this is only at the ∼ 1 𝜎 level. Increases in 𝐿 ∗ relative to the blank-field, as seen for both SMGs at 𝑧 ∼ 2 3 (and for their combined lu-. minosity function), imply that their environments may preferentially harbour brighter galaxies than those in the field. ALESS 5.1 exibits an offset of &lt; 𝜎 1 relative to the blank-field luminosity function at 𝑧 ∼ 3 3. .

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00063v1-figure-3a1d7c2d-7b28-4025-b17a-6daf86e8d2ef.jpg)

SMGs on average at 𝑧 ∼ 2 3) are consistent with being protoclusters, . albeit at the lower-density end. For the remainder of the analyses we assume that members of these overdensities will form larger structures by 𝑧 = 0, although we caution that the significance of these overdensities is relatively low (1.3-2.3 𝜎 𝛿 ) and thus it is uncertain whether they will coalesce by 𝑧 ∼ 0 (e.g. Chiang et al. 2013; Overzier et al. 2009; Angulo et al. 2012).

## 3.3 Spatial distribution of line emitters

In order to investigate the role of environment in shaping the evolution of SMGs, and to assess whether the target SMGs reside in special regions within any surrounding structures, we next explore the spatial distributions of coeval line emitters across our HAWK-I pointings. Due to the small size of the [Oiii] emitter sample, we limit this part of the analysis to the H 𝛼 emitters around ALESS 75.2 and 102.1. Figure 7 compares the surface density of H 𝛼 emitters as measured in annuli centred on each target SMG (where the density calculations account for masked and unobserved regions by assuming the density is the same as in the observed regions) with the surface densities one would expect based on the blank-field luminosity function from Sobral et al. (2013). The annuli have inner and outer radii increasing in increments of 2 . ′ 0, and are represented by dashed circles in Figure 4. Note that these large annuli are necessary due to the sample sizes, but make it difficult to probe the protocluster structures in detail. We therefore also show in Figure 8 the result of smoothing the distributions of H 𝛼 emitters in Pointing 5+75 (left) and Pointing 102 (right) using a 2D Gaussian kernel with width corresponding to 0.5 Mpc at 𝑧 ∼ 2 3. This . method of visualisation clarifies where the SMGs lie in relation to any density peaks and can highlight any substructures. For simplicity we assume that there are no H 𝛼 emitters in the unobserved region between the detector chips, or in regions of the image that have been masked, and thus the densities shown in these regions are potentially underestimated.

For ALESS 102.1, there is a noticeable decrease in the surface density of H 𝛼 emitters as a function of radial distance from the SMG, with the innermost bin in Figure 7 being significantly overdense relative to the field despite the environment not being overdense as a whole (see also §3.2). This is also clear from Figure 8, which shows that the SMG lies ∼ 20 ′′ from a ∼ 3 25 ( . ′ ∼ 1 6 Mpc) density . peak. Furthermore, Figure 4 demonstrates that the innermost 2 0 . ′ annulus contains more than half of the H 𝛼 emitters detected across the Pointing 102 pointing. Figure 8 includes a panel showing the location of the SMGs and our H 𝛼 emitter density maps in the context of the overdensity of Ly 𝛼 emitters (LAEs) at 𝑧 ∼ 2 3 mapped by . Yanget al. (2010). This shows that the small-scale overdensity around ALESS 102.1 is in a broader underdense region, and it is therefore unlikely to be a condensed infalling knot within a larger structure.

In the case of ALESS 75.2 we show two results in Figure 7: one where we include all H 𝛼 emitters in the pointing (open squares), and one where we exclude the dense clump of H 𝛼 emitters in the north-east (filled squares; see §2.5). In both cases, there is no significant trend in the H 𝛼 surface density as a function of separation from the SMG, although it does show signs of decreasing at the outermost radii if the dense north-easterly clump is excluded. This lack of trend implies that ALESS 75.2 does not reside in a particularly special region of the structure, and/or the structure extends beyond the HAWK-I field of view. The latter hypothesis is supported by the comparison of the H 𝛼 emitter overdensity with that of the LAEs from Yang et al. (2010) (Figure 8, right), which shows that the whole structure around ALESS 75.2 is within a larger region of LAE overdensity. This suggests that the H 𝛼 emitter structure likely spans a physical distance ≳ 3 5 Mpc at . 𝑧 ∼ 2 3, which is consistent with . the simulations of e.g. Muldrew et al. (2015), in which protoclusters are expected to extend over ≳ 10 Mpc at 𝑧 ∼ 2. The Yang et al. (2010) structure in this region includes the Ly 𝛼 blob CDFS-LAB03, which coincides with seven H 𝛼 emitters (see also §2.5). The overall picture is consistent with previous findings, in which Ly 𝛼 blobs are found to be associated with massive dark matter halos and filamentary large-scale structures (e.g. Geach et al. 2016; Umehata et al. 2019).

## 3.4 SMGcompanions: SFRs and stellar masses

In this section we investigate the dust-corrected star formation rates (SFR) and stellar masses ( 𝑀 ★ ) of the individual galaxies around each target SMG, to determine whether they lie on the main sequence of star formation at their epochs. This correlation between SFR and 𝑀 ★ has been observed out to 𝑧 ∼ 6 (e.g. Brinchmann et al. 2004; Elbaz et al. 2007; Daddi et al. 2007; González et al. 2010; Speagle et al. 2014; Schreiber et al. 2015; Scoville et al. 2017) and galaxies significantly above the main sequence are usually considered to be short-lived starbursts, whereas those significantly below the main sequence are typically quenched. The position of galaxies relative to the main sequence provides insights into their evolutionary state and can be used to infer the role of any environmentally-driven mechanisms enhancing or inhibiting star formation activity. Note that while ALESS 5.1 does not appear to reside in an overdense structure, the properties of the coeval [Oiii]+H 𝛽 emitters in its vicinity are still of interest and we thus include them in this part of the analysis.

We obtain stellar masses and SFRs for our galaxies by using the SED fitting code, magphys (da Cunha et al. 2008), to fit SEDs to the same fixed-aperture photometry used to derive photometric redshifts in §2.5. Figure 9 compares the relationship between SFR and stellar

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00063v1-figure-bb7becbc-e699-4d70-881e-47f3fbdcb02a.jpg)

∼

Figure 7. Surface density of H 𝛼 emitters measured in 2 . ′ 0 annuli centred on the two target SMGs at 𝑧 ∼ 2 3 and compared with expected values from the . blank-field H 𝛼 luminosity function (horizontal line and shaded region; Sobral et al. 2013). The shape of the field and positions of the SMGs means that coverage is incomplete with data for 74% (81%) of the inner, 36% (31%) of the middle, and 26% (27%) of the outer annuli for Pointing 5+75 (Pointing 102). Opensymbolsshowthevalues calculated if the dense clump of H 𝛼 emitters in the northeast of Pointing 5+75 (see §2.5) is included. Both 𝑧 ∼ 2 3 SMGshave . high densities of H 𝛼 emitters in the central ∼ 1 Mpc. For ALESS 102.1 the density falls with increasing separation from the SMG, though no significant trend exists for ALESS 75.2. The existence of a significant overdensity within ∼ 2 ′ of ALESS 102.1 with no evidence of further extension suggests it may reside in an early galaxy group, while ALESS 75.2 appears to reside in a larger structure that extends beyond the HAWK-I coverage.

mass for the H 𝛼 and [Oiii]+H 𝛽 emitters that are SMG companions with the main-sequence at similar epochs using the prescription from Speagle et al. (2014). These galaxies generally scatter about the main sequence at their respective epochs, following a similar trend of increasing SFR with increasing stellar mass. We thus find no significant evidence of enhanced star formation in any of these SMG environments, despite the range of overdensities that they span; this is contrary to some previous studies in which enhanced SFRs have been found in overdense environments at 𝑧 ≳ 1 (e.g. Elbaz et al. 2007; Cooper et al. 2008; Lemaux et al. 2022), however it is consistent with several other studies in which no environmentally-driven SFR enhancement is observed at high redshift (e.g. Scoville et al. 2013; Darvish et al. 2016; Zavala et al. 2019).

In addition to the magphys-derived SFRs, we also calculate the dust-corrected star formation rates for the H 𝛼 emitters using the scaling relation from Kennicutt (1998), modified for a Chabrier (2003) IMF:

$$\text{SFR} ( M _ { \odot } \, y r ^ { - 1 } ) = 4. 6 5 \times 1 0 ^ { - 4 2 } L _ { H \alpha } \, ( \text{erg s} ^ { - 1 } ) \,, \quad \quad ( 9 ) \quad \text{quadapplic} \alpha$$

where the H 𝛼 line flux has been corrected for contamination by the nearby [Nii] doublet (see §3.1.3) and we assume a dust attenuation of 𝐴 H 𝛼 = 1 mag (§3.1.5). For the [Oiii]+H 𝛽 emitters we assume an attenuation of 𝐴 [Oiii]+H 𝛽 = 1 35 mag following Khostovan et al. . (2015) and use the relation between SFR and 𝐿 [Oiii]+H 𝛽 from Osterbrock &amp; Ferland (2006), similarly modified for a Chabrier (2003) IMF:

$$S F R ( M _ { \odot } \, y r ^ { - 1 } ) = 4. 3 2 \times 1 0 ^ { - 4 2 } L _ { [ O m i ] + H \beta } \, ( \arg s ^ { - 1 } ) \,. \quad \ \ ( 1 0 ) \quad \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \$$

The bottom panel of Figure 9 shows how the ratio of the magphys- derived and line-derived SFR estimates varies with stellar mass. Also shown is the H 𝛼 dust attenuation required for the SFR derived from Equation 9, SFR H 𝛼 , to agree with the magphys-derived value, SFRmagphys , as given by:

$$\Big | \quad \ A _ { H \alpha } = 2. 5 \log _ { 1 0 } \left ( \frac { S F R _ { \text{MAGPHYS} } } { S F R _ { H \alpha } } \right ) + 0. 4 \,.$$

An analagous equation for 𝐴 [Oiii]+H 𝛽 can be obtained by adding 0.14 mag. It is evident that as one moves to higher stellar mass, the assumption that 𝐴 H 𝛼 ( 𝐴 [Oiii]+H 𝛽 ) = 1 0 . ( 1 35 . ) mag results in underestimated SFRs compared with the results from SED fitting. We therefore caution that while such an assumption may be suitable for galaxies with low-to-average stellar mass, it becomes less reliable for high-mass galaxies.

We also include the SMGs themselves in Figure 9, with the SFRs and stellar masses for these calculated by Danielson et al. (2017) and Birkin et al. (2021) using magphys. As expected for sources selected due to their infrared-brightness, the SMGs are among the most active galaxies in the observed fields. ALESS 102.1 is also one of the most massive galaxies in its environment, which suggests that if it is in a protocluster then it may be brightest cluster galaxy (BCG) progenitor, i.e. a proto-BCG. Similarly, ALESS 5.1 is massive relative to other galaxies in the surrounding region, but given the low density of this environment we deem it unlikely that this SMG is a proto-BCG. Conversely, ALESS 75.2 has a lower mass, which is not exceptional for its environment, and which points towards it being more likely to evolve into a normal cluster member. This is consistent with the spatial analysis of H 𝛼 emitters and LAEs (§3.3), which showed that ALESS 75.2 is offset from the densest regions of this field.

Since ALESS 102.1 has a counterpart H 𝛼 emitter in our sample, we also compare our magphys-derived SFR and stellar mass with those derived by Danielson et al. (2017). Our stellar mass of log ( 𝑀 ★ / M ⊙) = 11 49 . + 0 18 . -0 05 . is in good agreement with their value of log ( 𝑀 ★ / M ⊙) = 11 42 . + 0 06 . -0 06 . . Conversely, our SFR of log SFR M ( / ⊙ yr -1 ) = 2 75 . + 0 22 . -0 25 . is significantly higher than their estimate of log ( SFR M / ⊙ yr -1 ) = 2 12 . + 0 25 . -0 23 . . This is likely due to the inclusion of FIR and radio photometry in their SED fitting which are absent from our own fit; the dust component (and thus the SFR) is better constrained in the Danielson et al. (2017) SED fit. We therefore opt to use their values of SFR and stellar mass for this galaxy instead of our own.

## 3.5 Stellar mass functions

We next construct the stellar mass functions of the galaxies around each SMG and compare these with the blank field. The stellar mass functions are derived following a similar procedure as for the luminosity functions (see §3.1), minus the corrections that are only relevant to luminosity functions (dust attenuation, line flux contamination, filter profile corrections). Completeness corrections were applied to each mass bin according to the completeness values estimated in §3.1.2 based on the emission line fluxes. We then fit Schechter functions to the data:

$$\underset { \text{$n$ et al.} } { \text{ssume} } \quad \Phi ( M _ { * } ) d M _ { * } = \ln ( 1 0 ) \Phi ^ { * } \left ( \frac { M _ { * } } { M ^ { * } } \right ) ^ { \alpha + 1 } e ^ { - ( M _ { * } / M ^ { * } ) } d \log M _ { * } \,, \quad ( 1 2 )$$

where Φ ( 𝑀 ★ ) is the number density at stellar mass 𝑀 ★ , Φ ∗ is the normalisation of the stellar mass function, 𝑀 ∗ is the characteristic stellar mass, and 𝛼 is the slope at the faint end of the stellar mass function. Mass bins that are less than 50 percent complete are excluded from the fitting procedure. As with the luminosity functions, we also fix

Figure 8. Maps showing the variation of surface overdensity, 𝑛 𝑛 / field , of H 𝛼 emitters across Pointing 5+75 (left) and Pointing 102 (middle) fields, and in context of Ly 𝛼 emitter (LAE) density at 𝑧 ∼ 2 3 in the wider ECDFS (right). The maps are smoothed using a 2D Gaussian kernel with a width of 0.5 Mpc. Contour . level are 𝑛 𝑛 / field = 0 25 . , 0 5 . , 1 and increasing in intervals of 1 thereafter; dashed lines represent underdensities. Crosses mark the positions of the target SMGs, ALESS 75.2 and ALESS 102.1 and hatching indicates regions outside out HAWK-I coverage (including chip-gaps) or that are masked (e.g. due to the presence of a bright star). Note that the smoothing implicitly assumes that no H 𝛼 emitters reside in these regions, such that the densities here are conservative lower limits. Both SMGs are in/near H 𝛼 density peaks, although ALESS 75.2 is not in the highest density region in P05\_75. The rightmost panel shows the H 𝛼 overdensities in the two SMG fields compared to the wider LAE density measured in the whole (Yang et al. 2010); LAE contour levels are at 𝑛 𝑛 / field = 0 3 . , 0 5 . , 1 , 2 and 3. The region of highest H 𝛼 density in Pointing 5+75 corresponds to strong overdensity of LAEs, which contains the Ly 𝛼 blob CDFS-LAB03 (Yang et al. 2010, see §2.5) and there is an overall correlation between the H 𝛼 and LAE overdensities in this region. Conversely, despite being in a small region of localised H 𝛼 overdensity, ALESS 102.1 is in a region that is underdense in LAEs on the scales probed by Yang et al. (2010).

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00063v1-figure-5d2dd868-a687-4b3c-b38b-b17ea714cd1c.jpg)

the faint-end slope 𝛼 to the values derived for blank-field stellar mass functions by Sobral et al. (2014) and Khostovan et al. (2016) (i.e. 𝛼 = -1 37 and . 𝛼 = -1 3 for the for the H . 𝛼 and [Oiii]+H 𝛽 emitters, respectively). For [Oiii]+H 𝛽 emitters we also fix the characteristic stellar mass, 𝑀 ∗ , to the blank-field value of log ( 𝑀 ∗ / 𝑀 ⊙) = 10 96 . (Khostovan et al. 2016).

Table 5. Summary of the best-fit stellar mass function parameters for the companion galaxies in the environments of the target SMGs. In all cases, the faint-end slope of the stellar mass function, 𝛼 , is fixed to the value from the relevant blank-field stellar mass function (Sobral et al. 2014; Khostovan et al. 2016). Where indicated, the characteristic stellar mass, 𝑀 ∗ , is also fixed to the blank-field value from Khostovan et al. (2016).

The stellar mass functions are presented in Figure 10, with the parameters in Table 5. Uncertainties in each parameter are estimated following the same procedure as for those of the luminosity functions (see §3.1.7) and the correlation between the parameters and their uncertainties are shown in Figure 11, which demonstrates that at the upper limit the characteristic stellar mass, 𝑀 ∗ , is poorly constrained for all of our samples except the H 𝛼 emitters around ALESS 75.2. However, the lower limit is sufficient to show that in the 𝑧 ∼ 2 3 . SMG regions the characteristic stellar mass is significantly higher than the 𝑧 ∼ 2 3 field, which suggests that the stellar mass build-up . in SMG companion galaxies is further advanced than the coeval field (e.g. Muzzin et al. 2013). Due to our selection of H 𝛼 emitters the galaxies have non-negligible star-formation rates (though many are below the main sequence; §3.4). Observations using a local galaxy density estimator suggest that local environment has minimal effect on the stellar mass function of star-forming or quiescent galaxies at 𝑧 = 1 5-2 (Papovich et al. 2018). However, there is evidence of . protocluster environments being skewed towards containing galaxies with higher masses than the field (e.g. Cooke et al. 2014), consistent with our results.

## 3.6 Dark matter halo masses and evolution

We next estimate the total halo masses of the SMG environments in order to place them within the context of existing protoclusters and trace their likely evolution, focusing primarily on the overdense environments of the two SMGs at 𝑧 ∼ 2 3. Since these overdensities . are unvirialised and lack a detectable intracluster medium, the classic methods for weighing galaxy clusters cannot be used. Instead, we use

| SMG environment   | log ( 𝑀 ∗ / 𝑀 ⊙ )         | log ( Φ ∗ / Mpc - 3 )      |
|-------------------|---------------------------|----------------------------|
| ALESS 75.2        | 11 . 69 + 0 . 41 - 0 . 15 | - 3 . 37 + 0 . 09 - 0 . 25 |
| ALESS 102.1       | 12 . 08 + 0 . 52 - 0 . 59 | - 4 . 01 + 0 . 23 - 0 . 41 |
| SMGs at 𝑧 ∼ 2 . 3 | 11 . 85 + 0 . 30 - 0 . 34 | - 3 . 64 + 0 . 16 - 0 . 18 |
| ALESS 5.1         | 10 . 96 (fixed)           | - 3 . 73 + 0 . 08 - 0 . 08 |

two methods that have been used in protocluster studies, though the underlying assumptions required can lead to significant uncertainties, as discussed in the following subsections. The first method is detailed in §3.6.1 and uses the stellar-to-halo mass relation (SHMR) to estimate the high-redshift mass of the clusters (hereafter SHMR method) and evolve it to the local Universe using the Millennium and Millennium-II simulations (McBride et al. 2009; Fakhouri et al. 2010). The second method follows Steidel et al. (1998) and assumes the region of interest is a homogeneous sphere undergoing spherical collapse and uses the overdensity parameter to estimate the 𝑧 = 0 descendant mass, which we trace back to high-redshift using the Millennium and Millenium-II simulations. This is referred to as the spherical collapse model (SCM) method and detailed in §3.6.2. In §3.6.3 we discuss the evolution of the SMG environments compared with other systems and previous measurements.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00063v1-figure-0ff2d1c0-478d-4c23-97fd-a875088d29ba.jpg)

/circledot

Figure 9. (Top) magphys-derived SFR vs stellar mass for the H 𝛼 and [Oiii] emitters identified in this study, compared with the 𝑧 ∼ 2 3 and . 𝑧 ∼ 3 3 main . sequence (shaded regions represent 0.2 dex scatter; Speagle et al. 2014). The target SMGs are also shown, with masses and SFRs from Danielson et al. (2017) and Birkin et al. (2021) (black points; ALESS 102.1 is connected with a black dotted line to the counterpart H 𝛼 emitter identified from our data; discussed in §3.4). Dashed, dot-dashed, and dot-dot-dashed horizontal lines correspond to the minimum SFR sensitivity of our survey in the environments of ALESS 5.1, 75.2, and 102.1, respectively, based on estimates using the line luminosities (Equation 9). The galaxies generally follow the main sequence for their respective epochs, with some scatter in all three SMG environments. (Bottom) Ratio of magphys-derived SFRs to the SFRs derived from line luminosities using fixed dust H 𝛼 ([Oiii]) dust attenuations of 1.0 (1.35) mag, plotted as a function of stellar mass. The righthand axis shows the corresponding dust attenuation required to make the line luminosity-derived SFR match the magphys-derived SFR for H 𝛼 emitters, 𝐴 H 𝛼 (Equation 11). (Analogous values for [Oiii] emitters, 𝐴 [ Oiii ] , can be obtained by adding 0.14.) The observed correlation suggests that assuming a constant dust attenuation for all H 𝛼 /[Oiii] emitters results in underestimated SFRs at high stellar masses; such an approximation should therefore be used with caution. The black cross in the bottom-right of each panel shows the mean parameter uncertainties.

## 3.6.1 The SHMR method for deriving halo masses

The SHMR method for estimating protocluster masses involves identifying the most massive galaxy in the structure and converting its stellar mass to a halo mass, and taking this to be the halo mass of the whole structure. This method has been employed in recent protocluster studies (e.g. Long et al. 2020; Calvi et al. 2021; Sillassen et al. 2022; Ito et al. 2023) and implicitly assumes that all member galaxies occupy a single halo at the observed redshift of the structure, which may not be the case if some of the galaxies are still infalling. Nevertheless, we deem this assumption preferable to the commonly-used alternative of estimating the halo masses of each individual galaxy and summing them together (e.g. Long et al. 2020; Calvi et al. 2021), which risks 'double-counting' overlapping dark matter halos to produce an overestimate of the structure halo mass.

Note that we perform this calculation even for ALESS 5.1 and its surrounding [Oiii] emitters despite our analyses revealing no signs of their environment being significantly overdense. This is because this method does not explicitly depend on the density of the surrounding environment, and the high stellar mass of ALESS 5.1 (see §3.4 and 9) suggests it may yet reside in a massive halo. We do however caution that the result obtained here likely represents an extreme upper limit on the mass of any possible structure around ALESS 5.1.

We estimate halo masses for each target SMG and their candidate companion galaxies using the SHMR from Behroozi et al. (2013). We use the relation as defined at 𝑧 = 2 for galaxies in the environments of ALESS 75.2 and 102.1, and at 𝑧 = 3 for galaxies in the environment of ALESS 5.1. For the SMGs themselves we use the stellar masses from the literature (Danielson et al. 2017; Birkin et al. 2021, see also §3.4). Some of our galaxies have stellar masses which lie above the range at which the SHMR is defined and for these we use the stellarto-halo mass ratio for the largest halo mass at which the relation is defined (see Figure 7 of Behroozi et al. 2013) to convert the stellar mass to a halo mass. This affects only two H 𝛼 emitters from P5\_75, along with one from P102 which we have identified as a counterpart to ALESS 102.1 (see §3.4 and Figure 9). None of the [Oiii] emitters have stellar masses above the range for which the SHMR is defined at 𝑧 ∼ 3, but ALESS 5.1 does lie above this range. Uncertainties on individual galaxy halo masses are estimated based on the stellar mass uncertainties and the uncertainties in the SHMR derived by Behroozi et al. (2013).

We derive halo masses of log ( 𝑀 ℎ / M ⊙) = 11 45-14 46 for in-. . dividual H 𝛼 and [Oiii] emitters, with medians of log ( 𝑀 ℎ / M ⊙) = 12 16 . + 0 21 . -0 16 . , 11 94 . + 0 46 . -0 35 . , 11 75 . + 0 26 . -0 12 . for galaxies in the environments of ALESS 5.1, 75.2 and 102.1, respectively. The halo masses of the corresponding SMGs are log ( 𝑀 ℎ / M ⊙) = 13 94 . + 0 38 . -0 21 . , 12 02 . + 0 08 . -0 25 . , 14 39 . + 0 03 . -0 37 . , derived using their stellar masses reported in the literature (Danielson et al. 2017; Birkin et al. 2021). ALESS 5.1 and 102.1 are both the most massive galaxies in their respective environments; we therefore adopt their halo masses as the total masses of the potential structures at the observed redshifts. ALESS 75.2 is not the dominant galaxy in its environment, being surpassed in stellar (and hence inferred halo) mass by ∼ 40 percent of its companion H 𝛼 emitters. The most massive of these is a spectroscopically confirmed member (Popesso et al. 2009; Balestra et al. 2010) located in the H 𝛼 emitter density peak associated with the Ly 𝛼 blob CDFS-LAB03 (see Figure 8), with a halo mass of log ( 𝑀 ℎ / M ⊙) = 14 38 . + 0 01 . -0 40 . . We thus assume this is the total mass of the surrounding structure. Since high-redshift radio galaxies are commonly found in protocluster cores (e.g. Kurk et al. 2000; Venemans et al. 2002; Kuiper et al. 2011b; Wylezalek et al. 2013; Hayashi et al. 2012; Cooke et al. 2014), we search for radio counterparts for this galaxy in the second data release from the Very Large Array 1.4 GHz survey of the ECDFS (Miller et al. 2013), for which the typical sensitivity is 7.4 𝜇 Jy per 2 . ′′ 8 × 1 . ′′ 6 beam. We find no counterparts within 30 ′′ of this H 𝛼 emitter and thus rule it out as being a high-redshift radio galaxy.

The total halo masses at the observed redshift obtained using the SHMR method are thus log ( 𝑀 ℎ / M ⊙) = 13 94 . + 0 38 . -0 21 . , 14 38 . + 0 01 . -0 40 . , 14 39 . + 0 03 . -0 37 . for the environments of ALESS 5.1, 75.2, and 102.1, respectively. We note that these masses may be affected by systematic uncertainties on the stellar masses (due to uncertainties on star-formation histories and resulting massto-light ratios, which is particularly relevant for young starbursts; e.g. Wardlow et al. 2011) and on the SHMR for very high mass galaxies, which are present in the simulations from which the SHMR is derived (Behroozi et al. 2013). Indeed, predictions from halo mass functions

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00063v1-figure-324ed8c3-4b3b-4796-9487-051b4ff2cb77.jpg)

/circledot

/circledot

Figure 10. Stellar mass functions for the H 𝛼 ( left ) and [Oiii] ( right ) emitters identified in this study. The data are compared with blank-field studies of emission line galaxies at similar redshifts (Sobral et al. 2014; Khostovan et al. 2016) and shaded regions represent 1 𝜎 uncertainties. For each of our stellar mass functions, we fix the faint-end slope to the value derived for the blank field at a similar redshift: 𝛼 = -1 37 (Sobral et al. 2014) and . 𝛼 = -1 3 (Khostovan et al. 2016) for . the H 𝛼 and [Oiii]+H 𝛽 stellar mass functions, respectively. For the [Oiii]+H 𝛽 stellar mass function, we also fix the characteristic stellar mass to the blank-field value of log ( 𝑀 ∗ / 𝑀 ⊙ ) = 10 96 (Khostovan et al. 2016). For the H . 𝛼 emitters, the upper limit of 𝑀 ∗ is poorly constrained (see also Figure 11), which leads to large uncertainties at the high mass end. There are offsets between the blank field stellar mass functions and those around our SMGs in all targeted SMG regions; these are quantified in Figure 11 and Table 5.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00063v1-figure-c9150eb0-9466-4e47-b03f-7c0ae9bb3b82.jpg)

/circledot

Figure 11. Contours showing the correlated uncertainties on the Schechter fit parameters for the stellar mass functions shown in figure 10. In all cases, the faint-end slope, 𝛼 , is fixed to match to the blank-field stellar mass functions (Sobral et al. 2014; Khostovan et al. 2016), and for ALESS 5.1 M ∗ is also fixed. Symbols and contours have the same meaning as in Figure 6. For the H 𝛼 emitters at 𝑧 ∼ 2 3 the characteristic stellar mass, . 𝑀 ∗ , is effectively a lower limit due to the correlation with Φ ∗ ; this can also be seen in Figure 10. The offset between 𝑀 ∗ for field H 𝛼 emitters (Sobral et al. 2014) and the galaxies around 𝑧 ∼ 2 3 SMGs indicates that there is an excess of high-mass . galaxies around the SMGs, and this is likely partially responsible for the overdensity around ALESS 75.2.

suggest that the halo masses inferred from this method should be sufficiently rare that finding three such structures in the ∼ 0 25 deg 2 . ECDFS is unlikely (e.g. Press &amp; Schechter 1974; Tinker et al. 2008). Therefore, we consider these SHMR-derived halo masses to be upper limits and as such they are represented by the upper bounds on Figure 12 (the lower bounds are derived in §3.6.2), which compares the halo masses of the SMG environments with previously-studied galaxy clusters and protoclusters.

To assess whether these SMG environments are true protoclusters, we evolve the masses derived from the SHMR method to the present day masses and compare with known galaxy clusters in the local Universe. This is done using the redshift-dependent formula for the mean mass accretion rate derived from the results of the Millennium and Millennium-II simulations (McBride et al. 2009; Fakhouri et al. 2010):

$$\begin{array} { l l } & & \cdot \\ & & \langle \dot { M } \rangle _ { \text{mean} } = & 4 6. 1 M _ { \odot } y r ^ { - 1 } \left ( \frac { M _ { z } } { 1 0 ^ { 1 2 } M _ { \odot } } \right ) ( 1 + 1. 1 1 z ) \\ & & & \times \sqrt { \Omega _ { m, 0 } ( 1 + z ) ^ { 3 } + \Omega _ { \Lambda, 0 } } \,, \end{array}$$

where 𝑀 𝑧 is the halo mass of the structure at its observed redshift, and Ω 𝑚, 0 and ΩΛ , 0 are the present-day density parameters for matter and the cosmological constant according to our assumed cosmology (Planck Collaboration 2020).

For each overdensity we begin with the total halo masses estimated using the SHMR method and apply Equation 13 to incrementally add mass in small time steps until the present day is reached. The present-day masses obtained with this method are log ( 𝑀 ℎ,𝑧 = 0 / M ⊙) = 15 93 . + 0 62 . -0 33 . , 15 81 . + 0 01 . -0 55 . , 15 82 . + 0 04 . -0 50 . for the overdensities containing ALESS 5.1, ALESS 75.2, and ALESS 102.1, respectively. These masses suggest that these structures would all evolve into some of the most massive clusters in the Universe, rivalling that of the Coma cluster (e.g. Gavazzi et al. 2009; Ho et al. 2022) and other massive clusters at 𝑧 ≲ 1 such as those in the CLASH (Postman et al. 2012; Merten et al. 2015). However, given the rarity of such massive structures seen in the local Universe, we posit that the identification of progenitor structures around all three of our target SMGs is due to the systematics in the calculations, rather than a real occurrence. In addition to the possible systematics in the stellar masses and SHMR, as previously described, we also note that the mean mass accretion rate given by Equation 13 is poorly constrained for halos with masses of log ( 𝑀 ℎ / M ⊙) ≳ 14 beyond 𝑧 ∼ 0 5. Fur-. thermore, Equation 13 alone does not account for the diversity of evolutionary paths that real dark matter halos undergo, being the mean result for many halos in the Millennium simulation. Therefore, as for the high-redshift SHMR-derived halo masses, we also take these SHMR-derived 𝑧 = 0 and intermediate masses to be upper

limits. This upper limit on the halo mass at the SMG redshift and the evolution to the present day is shown in Figure 12 as the upper edges of the shaded regions for ALESS 75.2 and 102.1, and as a single solid line for ALESS 5.1.

## 3.6.2 The SCM method for deriving halo masses

An alternative method for estimating the present-day mass of each SMG environment is obtained following Steidel et al. (1998), which approximates each SMG environment as a homogeneous sphere undergoing spherical collapse. In this case the total present-day mass is given by:

$$M _ { h, z = 0 } & = \bar { \rho } V ( 1 + \delta _ { m } ) \,, & ( 1 4 ) & = 1 0$$

where ¯ 𝜌 is the mean comoving matter density of the Universe, 𝛿 𝑚 is the dark matter mass overdensity, and 𝑉 is the comoving volume of the structure. We refer to this method as the spherical collapse model (SCM) method. Since the assumption of spherical collapse is unphysical for environments that are not overdense, we only perform this calculation for the two SMG environments at 𝑧 ∼ 2 3. .

To estimate the volume of each overdensity, we assume that the structures are spherical and use the spatial extent of the structure on the sky to infer the angular diameter of the sphere containing the member galaxies. As discussed in §3.3, the overdensity around ALESS 75.2 extends beyond the confines of the HAWK-I pointing and therefore the size of the HAWK-I field-of-view can be used as a lower limit on the angular diameter of the structure. Therefore, for this environment we calculate the comoving volume for a spatial extent of ∼ 7 5 . ′ (Kissler-Patig et al. 2008), which corresponds to a comoving volume of 𝑉 ∼ 1000 cMpc 3 . This volume should be considered a lower limit, and thus the derived halo mass is also a lower limit. For the environment of ALESS 102.1, we assume that the structure is confined to the quadrant containing the SMG (§3.3), such that the angular diameter of the sphere is then ∼ 220 ′′ (Kissler-Patig et al. 2008), which gives a comoving volume of 𝑉 ∼ 110 cMpc 3 .

The dark matter mass overdensity is linked to the observed galaxy overdensity, 𝛿 𝑔 , via

$$\delta _ { m } & = \delta _ { g } / b \,, & ( 1 5 ) & \quad \text{obtain}$$

where 𝑏 is the bias parameter. To estimate the bias parameters for H 𝛼 emitters at 𝑧 ∼ 2 3, we make use of the relationship between . 𝑏 and 𝐿 H 𝛼 derived by Cochrane et al. (2017) at 𝑧 = 2 23. Following . a similar method to that of Stott et al. (2020), we derive a linear fit to this relation and estimate 𝑏 at the mean value of 𝐿 H 𝛼 for each SMG environment. This gives 𝑏 H 𝛼 = 2 9 . + 0 2 . -0 1 . and 2 8 . + 0 2 . -0 3 . for the candidate companions of ALESS 75.2 and 102.1, respectively.

Using the above values of 𝑏 and 𝑉 along with the 𝛿 𝑔 values calculated in §3.2, we obtain present-day halo masses of log ( 𝑀 ℎ,𝑧 = 0 / M ⊙) = 13 90 . + 0 09 . -0 10 . and 13 05 . + 0 12 . -0 14 . for the environments of ALESS75.2andALESS102.1,respectively. Contrary to the previous present-day mass estimates, these masses suggest that descendant of the environment of ALESS 102.1 is more akin to a galaxy group than a galaxy cluster (e.g. Han et al. 2015; Man et al. 2019), while the environment of ALESS 75.2 may evolve into a 'Virgo-like' or 'Fornax-like' cluster by 𝑧 = 0 (e.g. Chiang et al. 2013).

The significant disparity between the present-day halo mass estimates from the SCM method compared with the SHMR method likely arises from the assumptions and uncertainties in the calculations, including the systematics discussed in §3.6.1 and the estimates of the structure volumes. Note that for ALESS 75.2 the derived 𝑧 = 0 SCMhalo mass is a lower limit, due to the volume used being a lower limit. Since the masses derived from the SCM method are all lower

Figure 12. A comparison of protocluster halo masses across cosmic time. Two mass estimates are obtained for the environments of ALESS 75.2 and 102.1, at both the observed redshift of the potential structure and at 𝑧 = 0, using the methods described in §3.6. The evolutionary paths of these halos across cosmic time are estimated using the redshift-dependent mean mass accretion rate formula derived from the Millennium simulations (McBride et al. 2009; Fakhouri et al. 2010). The coloured shaded regions show the possible mass ranges and evolution for each of our target SMG environments, and are labelled with the ALESS ID of the inhabiting SMG. The upper bounds of these mass ranges correspond to the masses estimated using the stellarto-halo mass relations from Behroozi et al. (2013), while the lower bounds are derived by assuming a spherical collapse model (see text for details). For ALESS 5.1 only the former mass estimate is used, and is marked as an upper limit. The grey shaded region shows the expected evolution of a Coma-like cluster (Chiang et al. 2013) and coloured symbols show samples of clusters and protocluster from CLASH, GCLASS and CARLA, and protoclusters targeted due to their richness in DSFGs, as detailed in §3.6.3. We include regions showing measurements of SMG halo masses obtained from clustering studies (Hickox et al. 2012; Wilkinson et al. 2017; García-Vergara et al. 2020; Stach et al. 2021) and mark the borders between different gas regimes (Dekel &amp;Birnboim 2006). The two 𝑧 ∼ 2 3 SMGs reside in environments consistent . with protoclusters, although ALESS 102.1 may reside in a protogroup instead. Thelowerboundsofourmassestimatesare broadly consistent with the masses obtained from SMG clustering studies, while the upper bounds imply these halos may evolve into Coma-like structures or larger.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00063v1-figure-8634b9ba-f3e2-4cbb-a7d2-177c88722a03.jpg)

than the those from the SHMR method and its uncertainties, we proceed with the SCM estimates and adopt the lower bound of the 1𝜎 confidence intervals as lower limits for the present-day halo masses. Thus, the present-day halo mass estimates from the two methods gives a range of plausible evolutionary pathways for the two 𝑧 ∼ 2 3 . SMG environments as shown on Figure 12.

We also use the present-day halo mass estimates from the SCM method in combination with Equation 13 to trace the evolution of these SMG environments back in time to their observed redshifts, thereby obtaining a second estimate of the total mass at these redshifts. These masses are log ( 𝑀 ℎ / M ⊙) = 12 93 . + 0 07 . -0 08 . and 12 24 . + 0 10 . -0 11 . for the potential structures around ALESS 75.2 and ALESS 102.1, respectively. This calculation provides an evolutionary track that connects the lower halo mass limit at the 𝑧 ∼ 2 3 to the corresponding . value at 𝑧 = 0, and this is what defines the bottom edge of each shaded region in Figure 12.

Table 6. Halo mass estimates for each SMG environment.

| SMG         | log ( 𝑀 SHMR ℎ / M ⊙ ) 𝑎   | log ( 𝑀 SHMR ℎ,𝑧 = 0 / M ⊙ ) 𝑏   | log ( 𝑀 SCM ℎ / M ⊙ ) 𝑐   | log ( 𝑀 SCM ℎ,𝑧 = 0 / M ⊙ ) 𝑑   |
|-------------|----------------------------|----------------------------------|---------------------------|---------------------------------|
| ALESS 5.1   | 13 . 94 + 0 . 38 - 0 . 21  | 15 . 93 + 0 . 63 - 0 . 33        | -                         | -                               |
| ALESS 75.2  | 14 . 38 + 0 . 01 - 0 . 40  | 15 . 81 + 0 . 01 - 0 . 55        | 12 . 93 + 0 . 07 - 0 . 08 | 13 . 90 + 0 . 09 - 0 . 10       |
| ALESS 102.1 | 14 . 39 + 0 . 03 - 0 . 37  | 15 . 82 + 0 . 04 - 0 . 50        | 12 . 24 + 0 . 10 - 0 . 11 | 13 . 05 + 0 . 12 - 0 . 14       |

- 𝑎 Halo mass derived using the SHMR method (§3.6.1) at the redshift of the SMG (i.e. 𝑧 ∼ 2 3 for ALESS 75.2 and ALESS 102.1, and . 𝑧 ∼ 3 3 for ALESS 5.1). . As discussed in §3.6.1, we consider these to be upper limits.
- 𝑏 Halo mass derived using the SHMR method and evolved to 𝑧 = 0 using Equation 13; these masses are considered to be upper limits (see §3.6.1).
- 𝑐 Halo mass derived using the SCM method and traced back to the SMG redshift using Equation 13 (§3.6.2).
- 𝑑 Descendant halo mass at 𝑧 = 0 derived using the SCM method (§3.6.2).

## 3.6.3 Halo masses and evolution

The halo mass estimates of each SMG environment are summarised in Table 6, and Figure 12 compares the SMG halo masses and their evolution with previously-studied galaxy clusters and protoclusters. As described in §3.6.1 and §3.6.2, the upper and lower bounds shown for the SMG halos are derived from the masses calculated using the SHMR method and the lower limits from the SCM method, respectively (with the exception of ALESS 5.1, for which only the SHMR method is used). Thus, this region encompasses the full range of possible halo masses and evolution for the SMGs. In Figure 12 these are comparedwith clusters from the Cluster Lensing And Supernova Survey with Hubble (CLASH; Postman et al. 2012; Merten et al. 2015); clusters from the Gemini Cluster Astrophysics Spectroscopic Survey (GCLASS;Muzzinet al. 2012; van der Burg et al. 2014); clusters and protoclusters from the Clusters Around Radio-Loud AGN program (CARLA; Wylezalek et al. 2013; Mei et al. 2022). We also show the halo masses calculated by Casey (2016) for overdense structures known to be rich in dusty star-forming galaxies (DSFGs), including: the GOODS-N protocluster at 𝑧 = 1 99 (Blain et al. 2004; Chapman . et al. 2009); the COSMOS protoclusters at 𝑧 = 2 10 (Yuan et al. 2014) . and 𝑧 = 2 47 (Casey et al. 2015; Diener et al. 2015; Chiang et al. . 2015); the 'Spiderweb' protocluster, MRC 1138-256, at 𝑧 = 2 16 . (Kurk et al. 2000; Kuiper et al. 2011a); the SSA 22 protocluster at 𝑧 = 3 09 (Steidel et al. 1998; Hayashino et al. 2004; Tamura et al. . 2009; Lehmer et al. 2009; Umehata et al. 2015); the GN20 overdensity at 𝑧 = 4 05 (Daddi et al. 2009; Hodge et al. 2013b). In this . sample of DSFG-rich protoclusters we additionally include the halo mass of the DRC at 𝑧 = 4 (Long et al. 2020). Figure 12 shows that the potential structures surrounding ALESS 75.2 and 102.1 are consistent with being protoclusters at 𝑧 ∼ 2 3. The upper limit for the halo . mass derived for ALESS 5.1 is also consistent with this environment being a protocluster, but we emphasise that our previous analyses suggest it is unlikely to reside in such a structure.

Both the present-day and high-redshift masses obtained using the SHMR method (§3.6.1) are significantly higher than those obtained using the SCM method (§3.6.2), and higher than expected for three structures all found in a survey of ∼ 0 25 deg 2 . based on predictions from halo mass functions (e.g. Press &amp; Schechter 1974; Tinker et al. 2008). This is consistent with the SHMR masses being affected by systematic effects that make them upper limits, as discussed in §3.6.1. The SCM-derived masses are typically lower than the masses of protoclusters at similar redshifts (see Figure 12), but are consistent with the results of SMG clustering studies (Hickox et al. 2012; Wilkinson et al. 2017; García-Vergara et al. 2020; Stach et al. 2021, see Figure 12), which generally agree that SMGs reside in halos with log ( 𝑀 ℎ / M ⊙) ≲ 13 at 𝑧 = 1-3. However, we note that the studies by Hickox et al. (2012) and Wilkinson et al. (2017) were both conducted using single-dish observations and are thus limited by false counterpart identification and source blending. Although the highresolution interferometric studies by García-Vergara et al. (2020) and Stach et al. (2021) are not afflicted by these limitations, they present conflicting results for the halo masses of the SMG population, likely stemming from the methodological differences described in §1.

Figure 12 also includes the approximate boundaries separating different gas regimes, as proposed by Dekel &amp; Birnboim (2006): in halos for which log ( 𝑀 ℎ / M ⊙) ≲ 12, inflowing gas is predominantly cold and enables the growth of galaxies; in halos above this mass threshold, these gas inflows are shock-heated resulting in strangulation of the galaxy within. However, if these massive halos still fall below some other, redshift-dependent mass threshold (as marked by the 'cold in hot' boundary in Figure 12), then penetrating cold gas may still be sustaining galaxy growth. At 𝑧 ∼ 2 3 this mass threshold . is log ( 𝑀 ℎ / M ⊙) ∼ 12 9, while at . 𝑧 = 3 3 it is log . ( 𝑀 ℎ / M ⊙) ∼ 14 0. .

Based on our mass estimates, the halo of ALESS 5.1 is likely in the 'cold in hot' category at its observed redshift, particularly when noting that the mass of this halo is possibly overestimated. It is therefore probable that ALESS 5.1, along with any other galaxy that may share its halo, is undergoing growth sustained by penetrating cold gas inflows. Conversely galaxies in the environment of ALESS 75.2 are more likely to be undergoing strangulation due to shock-heating in the halo at the observed redshift; the SHMR-derived halo mass lies significantly above the limit for 'cold in hot' gas inflows, and the 1 𝜎 confidence interval for the SCM-derived mass only just crosses below the limit. We cannot conclude anything about the gas regime in the environment of ALESS 102.1 as the 'cold in hot' boundary is straddled by the mass estimates for this structure.

## 4 CONCLUSIONS

We have conducted a wide-field narrowband survey of star-forming galaxies in the environments of three SMGs at 𝑧 ∼ 2 3 and . 𝑧 ∼ 3 3 . to determine whether these SMGs reside in protocluster-like environments. By studying individual SMGs selecting based only on their spectroscopic redshifts we have measured 'typical' SMG environments. Our main conclusions are as follows:

- · Using HAWK-I Br 𝛾 and 𝐾 𝑠 photometry, we identified a total of 147 candidate emission line galaxies in the two HAWK-I pointings containing the three target SMGs. After extracting photometry from archival UV-to-NIR broadband images, we performed SED fitting with eazy-py to obtain photometric redshifts for these galaxies and identified 44, 11, and 4 companion galaxies to the SMGs ALESS 75.2

( 𝑧 spec = 2 294), . ALESS 102.1 ( 𝑧 spec = 2 296), . and ALESS 5.1 ( 𝑧 spec = 3 303), respectively. .

- · By constructing luminosity functions for each SMG environment and comparing with blank-field luminosity functions from the literature at similar redshifts, we measure overdensity parameters of 𝛿 𝑔 = 0 2 . + 2 5 . -0 7 . , 2 6 . + 1 4 . -1 2 . , and 0 2 . + 0 6 . -0 5 . across the whole ∼ 4 Mpc HAWK-I field of view for ALESS 5.1, 75.2, and 102.1, respectively. Whilst ALESS 102.1 is not overdense on these large scales, it does sit in a ∼ 1 Mpc region with 𝛿 𝑔 = 3 8 . + 2 4 . -1 8 . . Therefore 2/3 of the target SMGs reside in overdense environments.
- · We considered the spatial distribution of companion H 𝛼 emitters in the environments of the two SMGs at 𝑧 ∼ 2 3 (ALESS 75.2 . and 102.1) by measuring their density in annuli around the SMGs and by constructing overdensity maps. For ALESS 75.2 the companion galaxies are spread out across the entire HAWK-I field of view, spanning a few Mpc. This is consistent with simulations, in which protoclusters are seen to extend over several Mpc at 𝑧 ∼ 2-3. The SMG resides near a possible density peak of H 𝛼 emitters, although a greater peak is seen a few arcminutes eastward which appears to be associated with a previously discovered Ly 𝛼 blob (Yang et al. 2010). The overdensity around ALESS 102.1 is smaller ( ∼ 1 Mpc) and could instead evolve into a galaxy group locally.
- · Stellar masses and SFRs were obtained for the companion galaxies in each SMG environment by performing SED fitting with magphys. The galaxies are generally scattered about the star-forming main sequence at their respective epochs, with no evidence of enhanced star formation activity in either environment.
- · Two methods were used to estimate the total halo mass of each of the two overdense SMG environments, which provided upper and lower bounds on the halo masses at the observed redshifts and evolving to the present day. These reveal that ALESS 75.2 likely resides in a protocluster, while ALESS 102.1 resides in either a protocluster or a protogroup.
- · We therefore surmise that 2/3 of these SMGs are strong candidates for the progenitors of massive elliptical galaxies in clusters, although the possibility remains for them to end up in galaxy groups. If these targets are indeed representative of 'typical' SMGs then this suggests that SMGs in general are likely to evolve into massive elliptical galaxies by the present day, as suggested by certain evolutionary models (e.g. Sanders et al. 1988; Hopkins et al. 2008), but with significant variation in the surrounding environments.

With this study we have demonstrated the efficacy of narrowband surveys as a means of searching for galaxy overdensities around SMGsselected without bias towards particular environments. Future followup with larger samples of SMGs and/or spectroscopic confirmation of companion galaxies would confirm the nature of the overdensities that we have detected, and resolved analyses (e.g. with ALMA and/or JWST) will further reveal how the member galaxies evolve.

## ACKNOWLEDGEMENTS

We are extremely grateful to Ian Smail for numerous insightful discussions, which significantly improved this work. The research is based on observations collected at the European Southern Observatory under ESO programme 0103.A-0668(A). TC received support from the Science and Technology Facilities Council (STFC; 2287406) and the Faculty of Science and Technology at Lancaster University. JW acknowledges an STFC Ernest Rutherford Fellowship (ST/P004784/2). HD acknowledges support from the Agencia

Estatal de Investigación del Ministerio de Ciencia, Innovación y Universidades (MCIU/AEI) under grant (Construcción de cúmulos de galaxias en formación a través de la formación estelar oscurecida por el polvo) and the European Regional Development Fund (ERDF) with reference (PID2022-143243NB-I00/10.13039/501100011033). The authors would also like to acknowledge the publicly available software Topcat (Taylor 2005), as well as the Python packages NumPy (Harris et al. 2020), SciPy (Virtanen et al. 2020), Astropy (Astropy Collaboration et al. 2013, 2018), Matplotlib (Hunter 2007), and Photutils (Bradley et al. 2022).

## DATA AVAILABILITY

The data used in this publication can be accessed via the ESO Science Archive under Program ID: 0103-0668(A), PI: Wardlow.

## REFERENCES

| Angulo R. E., Springel V., White S. D. M., Cole S., Jenkins A., Baugh C. M., Frenk C. S., 2012, MNRAS, 425, 2722                                                                                                              |
|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Arsenault R., et al., 2008, in Hubin N., Max C. E., Wizinowich P. L., eds, Society of Photo-Optical Instrumentation Engineers (SPIE) Con- ference Series Vol. 7015, Adaptive Optics Systems. p. 701524, doi:10.1117/12.790359 |
| Astropy Collaboration et al., 2013, A&A, 558, A33                                                                                                                                                                             |
| Astropy Collaboration et al., 2018, AJ, 156, 123                                                                                                                                                                              |
| Balestra I., et al., 2010, A&A, 512, A12                                                                                                                                                                                      |
| Barger A. J., Cowie L. L., Sanders D. B., Fulton E., Taniguchi Y., Sato Y., Kawara K., Okuda H., 1998, Nature, 394, 248                                                                                                       |
| Barger A. J., Wang W. H., Cowie L. L., Owen F. N., Chen C. C., Williams J. P., 2012, ApJ, 761, 89                                                                                                                             |
| Baugh C. M., Cole S., Frenk C. S., Lacey C. G., 1998, ApJ, 498, 504                                                                                                                                                           |
| Baugh C. M., Lacey C. G., Frenk C. S., Granato G. L., Silva L., Bressan A., Benson A. J., Cole S., 2005, MNRAS, 356, 1191                                                                                                     |
| Behroozi P. S., Wechsler R. H., Conroy C., 2013, ApJ, 770, 57 Bertin E., 2006, in Gabriel C., Arviset C., Ponz D., Enrique S., eds, nomical Society of the Pacific Conference Series Vol. 351,                                |
| Astro- Astronomical Data Analysis Software and Systems XV. p. 112 Bertin E., 2010, SWarp: Resampling and Co-adding FITS Images                                                                                                |
| Together (ascl:1010.068) Bertin E., Arnouts S., 1996, A&AS, 117, 393                                                                                                                                                          |
| Béthermin M., Dole H., Lagache G., Le Borgne D., Penin A., 2011, A&A, 529, A4                                                                                                                                                 |
| Birkin J. E., et al., 2021, MNRAS, 501, 3926 Blain A. W., Smail I., Ivison R. J., Kneib J. P., Frayer D. T., 2002, Phys. Rep., 369, 111                                                                                       |
| Blakeslee J. P., et al., 2003, ApJ, 596, L143 Bleem L. E., et al., 2015, ApJS, 216, 27 Böhringer H., et al., 2001, A&A, 369, 826                                                                                              |
| Bond J. R., Kofman L., Pogosyan D., 1996, Nature, 380, 603                                                                                                                                                                    |
| Bothwell M. S., et al., 2013, MNRAS, 429, 3047 Bradley L., et al., 2022, astropy/photutils:, doi:10.5281/zenodo.6385735,                                                                                                      |
| Brammer G. B., van Dokkum P. G., Coppi P., 2008, ApJ, 686, 1503 Brinchmann J., Charlot S., White S. D. M., Tremonti C., Kauffmann                                                                                             |
| G., Heckman T., Brinkmann J., 2004, MNRAS, 351, 1151                                                                                                                                                                          |
| A. J., Warren S. J., Hewett P. C., Clements D. L., 1995, 273, 513                                                                                                                                                             |
| J., Sobral D., Stroe A., Best P., Smail I., Lehmer B., Harrison Thomson A., 2017, MNRAS, 464, 303                                                                                                                             |
| R., Dannerbauer H., Arrabal Haro P., Rodríguez Espinosa J. M., Tuñón C., Pérez González P. G., Geier S., 2021, MNRAS, 502, 4558                                                                                               |
| Calvi                                                                                                                                                                                                                         |
| Calzetti D., Armus L., Bohlin R. C., Kinney A. L., Koornneef J., Storchi- Bergmann T., 2000, ApJ, 533, 682                                                                                                                    |
| Muñoz-                                                                                                                                                                                                                        |
| Calhau                                                                                                                                                                                                                        |
| Bunker                                                                                                                                                                                                                        |
| https://doi.org/10.5281/zenodo.6385735                                                                                                                                                                                        |
| Blain A. W., Chapman S. C., Smail I., Ivison R., 2004,                                                                                                                                                                        |
| ApJ, 611, 725                                                                                                                                                                                                                 |
| MNRAS,                                                                                                                                                                                                                        |
| C.,                                                                                                                                                                                                                           |

| Casali M., et al., 2006, in McLean I. S., Iye M., eds, Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series Vol. 6269, Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series. p. 62690W, doi:10.1117/12.670150 Casey C. M., 2016, ApJ, 824, 36 C. M., Narayanan D., Cooray A., 2014, Phys. Rep., 541, 45 C. M., et al., 2015, ApJ, 808, L33 G., 2003, PASP, 115, 763 S. C., et al., 2005, ApJ, 622, 772 S. C., Blain A., Ibata R., Ivison R. J., Smail I., Morrison G., 2009,   |
|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Casey                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| Casey Chabrier                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| Chapman                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| Chapman ApJ, 691, 560                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| Cheng T., et al., 2019, MNRAS, 490, 3840                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| Chiang Y.-K., Overzier R., Gebhardt K., 2013, ApJ, 779, 127                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| Chiang Y.-K., et al., 2015, ApJ, 808, 37                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| Chiang Y.-K., et al., 2017, ApJ, 844, L23                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| Clements D. L., Dunne L., Eales S., 2010,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| Cochrane R. K., Best P. N., Sobral D., Smail J. E., 2017, MNRAS, 469, 2913                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| MNRAS, 403, 274 I., Wake D. A., Stott J. P., Cooke E. A., Hatch N. A., Muldrew S. I., Rigby E. E., Kurk J. D.,                                                                                                                                                                                                                                                                                                                                                                                                               |
| Geach 2014, MNRAS, 440, 3262                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| Cooper M. C., et al., 2008, MNRAS, 383, 1058 Coppin K., et al., 2006, MNRAS, 372, 1621                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| Cucciati O., et al., 2014, A&A, 570, A16 Daddi E., et al., 2007, ApJ, 670, 156                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| Daddi E., et al., 2009, ApJ, 694, 1517                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| Damen M., et al., 2011, ApJ, 727, 1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| Danielson A. L. R., et al., 2017, ApJ,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| Darvish B., Mobasher B., Sobral D.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| 840, N., Faisst A., Capak P., 2016, ApJ, 825, 113                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| 78 Rettura A.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| Scoville Davé R., Finlator K., Oppenheimer B. D., Fardal M.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| Katz Kereš D., Weinberg D. H., 2010, MNRAS, 404, 1355 Davies L. J. M., Bremer M. N., Stanway E. R., Husband K.,                                                                                                                                                                                                                                                                                                                                                                                                              |
| N., Lehnert M. D., Mannering E. J. A., 2014, MNRAS, 438, 2732 De Lucia G., Blaizot J., 2007, MNRAS, 375, 2                                                                                                                                                                                                                                                                                                                                                                                                                   |
| Dekel A., Birnboim Y., 2006, MNRAS, 368, 2 Diener C., et al., 2013, ApJ, 765, 109 Diener C., et al., 2015, ApJ, 802, 31 Dressler A., 1980, ApJ, 236, 351 Dudzeviči¯t˙ u e U., et al., 2020, MNRAS, 494, 3828                                                                                                                                                                                                                                                                                                                 |
| Eales S., Lilly S., Gear W., Dunne L., Bond J. R., Hammer F., Le Fèvre O., Crampton D., 1999, ApJ, 515, 518 Eddington A. S., 1913, MNRAS, 73, 359                                                                                                                                                                                                                                                                                                                                                                            |
| Elbaz D., et al., 2007, A&A, 468, 33                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| Ellis R. S., Smail I., Dressler A., Couch W. J., Oemler                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| H., Sharples R. M., 1997, ApJ, 483, 582                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| Augustus Fakhouri O., Ma C.-P., Boylan-Kolchin M., 2010, MNRAS,                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| Finkelstein S. L., et al., 2022, ApJ, 928, 52 García-Vergara C., Hodge J., Hennawi J. F., Weiss A.,                                                                                                                                                                                                                                                                                                                                                                                                                          |
| A. D., Hickox R., 2020, ApJ, 904, 2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| Garn T., et al., 2010, MNRAS, 402, 2017 Gavazzi R., Adami C., Durret F., Cuillandre J. C., Ilbert O., R., Ulmer M. P., 2009, A&A, 498, L33                                                                                                                                                                                                                                                                                                                                                                                   |
| Mazure Gawiser E., et al., 2006a, ApJS, 162, 1                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| Gawiser E., et al., 2006b, ApJ, 642, L13                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| Geach J. E., et al., 2005, MNRAS, 363, 1398                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| Geach J. E., Smail I., Best P. N., Kurk J., Casali M., 2008, MNRAS, 388, 1473                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| Ivison R. J., Geach J. E., et al., 2016, ApJ, 832, 37                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| Gilbank D. G., Gladders M. D., Yee H. K. C., Hsieh B. C., 2011, AJ,                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| 141, Gladders M. D., Yee H. K. C., 2000, AJ, 120, 2148                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| Gladders M. D., Yee H. K. C., 2005, ApJS, 157, 1                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| González V., Labbé I., Bouwens R. J., Illingworth G., Franx M., Kriek                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| 94 Brammer G. B., 2010, ApJ, 713, 115                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| Greenslade J., Clements D. L., Petitpas G., Asboth V., Conley A.,                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| Fournon I., Riechers D., 2020, MNRAS, 496, 2315                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| M., Pérez- Greve T. R., et al., 2005, MNRAS, 359, 1165                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| J., Butcher                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| 406, 2267                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| Myers                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| Wardlow                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| J.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| A.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| Coppin                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| Pelló K.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |

| Gullberg B., et al., 2019, MNRAS, 490, 4956 Hainline L. J., Blain A. W., Smail I., Alexander D. M., Armus L., Chapman S. C., Ivison R. J., 2011, ApJ, 740, 96 J., et al., 2015, MNRAS, 446, 1356 C. R., et al., 2020, Nature, 585, 357 M., et al., 2013, J. Cosmology Astropart. Phys., 2013, 008                                                                                                                                                                                                                                                                                         |
|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Han Harris                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| Hasselfield Hatch N. A., Kurk J. D., Pentericci L., Venemans B. P., Kuiper E., Miley G. K., Röttgering H. J. A., 2011, MNRAS, 415, 2993 Hayashi M., Kodama T., Tadaki K.-i., Koyama Y., Tanaka I., 2012, ApJ, 757, 15                                                                                                                                                                                                                                                                                                                                                                     |
| T., et al., 2004, AJ, 128, 2073                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| Hayashino C. C., et al., 2021, MNRAS, 502, 2922 J. P., Mullis C. R., Voges W., Böhringer H., Briel U. G., Gioia I. M., Huchra J. P., 2006, ApJS, 162, 304                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| R. C., et al., 2012, MNRAS, 421, 284                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| Hayward Henry Hickox                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| Hildebrandt H., et al., 2006, A&A, 452, 1121 Ho M., Ntampaka M., Rau M. M., Chen M., Lansberry A., Ruehle F., Trac H., 2022, Nature Astronomy, 6, 936 Hodge J. A., da Cunha E., 2020, Royal Society Open Science, 7, 200556 Hodge J. A., et al., 2013a, ApJ, 768, 91 Hodge J. A., Carilli C. L., Walter F., Daddi E., Riechers D., 2013b, ApJ, 776, 22 Hodge J. A., et al., 2016, ApJ, 833, 103 Hopkins P. F., Hernquist L., Cox T. J., Kereš D., 2008, ApJS, 175, 356                                                                                                                    |
| 2012, ApJS, 203, 23 Hughes D. H., et al., 1998, Nature, 394, 241 Hunter J. D., 2007, Computing in Science &Engineering, 9, 90 Ikarashi S., et al., 2015, ApJ, 810, 133 Ito K., et al., 2023, arXiv e-prints, p. arXiv:2301.08845 Ivison R. J., et al., 2000, ApJ, 542, 27 Ivison R. J., et al., 2013, ApJ, 772, 137 Kennicutt Robert C. J., 1998, ARA&A, 36, 189 Khostovan A. A., et al., 2015, MNRAS, 452, 3948 Khostovan A. A., Sobral D., Mobasher B., Smail I., Darvish B., Nayyeri H., Hemmati S., Stott J. P., 2016, MNRAS, 463, 2363 Kissler-Patig M., et al., 2008, A&A, 491, 941 |
| Koprowski M. P., Dunlop J. S., Michałowski M. J., Cirasuolo M., Bowler R. A. A., 2014, MNRAS, 444, 117 KoyamaY., KodamaT., Tadaki K.-i., Hayashi M., Tanaka M., Smail I., Tanaka I., Kurk J., 2013, MNRAS, 428, 1551 Kriek M., Conroy C., 2013, ApJ, 775, L16                                                                                                                                                                                                                                                                                                                             |
| R. G., 1980, ApJS, 43, 305                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| Kron                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| Kuiper E., et al., 2011a, MNRAS, 415, 2245 Kuiper E., et al., 2011b, MNRAS, 417, 1088 Kurk J. D., et al., 2000, A&A, 358, L1                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| C. G., Baugh C. M., Frenk C. S., Silva L., Granato G. L., Bressan A., 2008, MNRAS, 385, 1155 C. G., Baugh C. M., Frenk C. S., Benson A. J., Orsi A., Silva L.,                                                                                                                                                                                                                                                                                                                                                                                                                            |
| Lacey Lacey Granato G. L., Bressan A., 2010, MNRAS, 405, 2 Le Fèvre O., et al., 2005, A&A, 439, 845 Lehmer B. D., et al., 2005, ApJS, 161, 21                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| Lehmer B. D., et al., 2009, MNRAS, 400, 299 Lemaux B. C., et al., 2014, A&A, 572, A41                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| Hsieh B.-C., Wang W.-H., Hsieh C.-C., Lin L., Yan H., Lim J., Ho P. T. P.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| Lemaux B. C., et al., 2022, A&A, 662, A33                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| Long A. S., et al., 2020, ApJ, 898, 133                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| Lovell C. C., Geach J. E., Davé R.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| Q., 2021, MNRAS, 502, 772                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| Narayanan                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| MacKenzie T. P., et al., 2017, MNRAS, 468,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| Madau P., Dickinson M., 2014, ARA&A, 52,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| D.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| Magnelli B., et al., 2012, A&A, 539, A155                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| Man Z.-Y., Peng Y.-J., Shi J.-J., Kong X.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| 4006                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| 2019, ApJ, 881, 74                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| 415 Zhang                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| Dou J., Guo K.-X.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| C.-P., Matsuda Y., et al., 2004, AJ, 128, 569                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| Li                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |

| Michałowski M. J., Dunlop J. S., Cirasuolo M., Hjorth J., Hayward C. C., Watson D., 2012, A&A, 541, A85                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Michałowski M. J., et al., 2017, MNRAS, 469, 492                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| Miettinen O., et al., 2017, A&A, 606, A17 Miller N. A., et al., 2013, ApJS, 205, 13                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| Moorwood A. F. M., van der Werf P. P., Cuby J. G., Oliva E., 2000, A&A, 362, 9 S. I., Hatch N. A., Cooke E. A., 2015, MNRAS, 452, 2528                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| Muldrew Muzzin A., et al., 2009, ApJ, 698, 1934                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| Muzzin A., et al., 2012, ApJ, 746, 188 Muzzin A., et al., 2013, ApJ, 777, 18                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| Narayanan D., Hayward C. C., Cox T. J., Hernquist L., Jonsson P.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| Younger J. D., Groves B., 2010, MNRAS, 401, 1613 D., et al., 2015, Nature, 525, 496                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| S.-M., Somerville R. S., Ferguson H. C., Huang K.-H., Lotz J.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| Narayanan Niemi moer A. M., 2012, MNRAS, 421, 1539                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| Oke J. B., Gunn J. E., 1983, ApJ, 266, 713 Osterbrock D. E., Ferland G. J., 2006, Astrophysics of gaseous nebulae                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| active galactic nuclei. CA: University Science Books Overzier R. A., 2016, A&ARv, 24, 14 Overzier R. A., Guo Q., Kauffmann G., De Lucia G., Bouwens R.,                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| Koeke-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| G., 2009, MNRAS, 394, 577                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| and Lemson F., et al., 2016, A&A, 592, A2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| Pacaud Palunas P., Teplitz H. I., Francis P. J., Williger G. M., Woodgate B. E., 2004, ApJ, 602, 545 L., et al., 2021, MNRAS, 504, 928 C., et al., 2018, ApJ, 854, 30                                                                                                                                                                                                                                                                                                                                                                                                                         |
| Pantoni Papovich Paufique J., et al., 2010, in Ellerbroek B. L., Hart M., Hubin N., Wizinowich P. L., eds, Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series Vol. 7736, Adaptive Optics Systems II. p. 77361P, doi:10.1117/12.858261 Pirard J.-F., et al., 2004, in Moorwood A. F. M., Iye M., eds, Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series Vol. 5492, Ground-based Instrumentation for Astronomy. pp 1763-1772, doi:10.1117/12.578293 Planck Collaboration 2020, A&A, 641, A6 Planck Collaboration et al., 2016, A&A, 594, A27 |
| Postman M., et al., 2012, ApJS, 199, 25 Press W. H., Schechter P., 1974, ApJ, 187, 425 Ramsay S. K., Mountain C. M., Geballe T. R., 1992, 259, 751 Rowan-Robinson M., et al., 2018, A&A, 619, A169 Sanders D. B., Soifer B. T., Elias J. H., Madore B. F., Matthews                                                                                                                                                                                                                                                                                                                           |
| Pope A., et al., 2006, MNRAS, 370, 1185 Popesso P., et al., 2009, A&A, 494, 443 MNRAS,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| bauer G., Scoville N. Z., 1988, ApJ, 325,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| K., Neuge- Schechter P., 1976, ApJ, 203, 297                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| 74                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| Schreiber C., et al., 2015, A&A, 575, A74 Scoville N., et al., 2013, ApJS, 206, 3 Scoville N., et al., 2017, ApJ, 837, 150                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| Smolčić V., et al., 2017, A&A, 597, A4 Sobral D., et al., 2009, MNRAS, 398, 75 Sobral D., Best P. N., Matsuda Y., Smail I., Geach J. E., MNRAS, 420, 1926                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| Cirasuolo M.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| Smail I., Ivison R. J., Blain A. W., 1997, ApJ, 490, Smail I., Ivison R. J., Gilbank D. G., Dunlop J. S., Stevens J. A., 2003, ApJ, 583, 551 Smith D. J. B., Hayward C. C., Jarvis M. J., Simpson 471, 2453                                                                                                                                                                                                                                                                                                                                                                                   |
| Shimasaku K., et al., 2003, ApJ, 586, L111 Siebenmorgen R., Carraro G., Valenti E., Petr-Gotzens                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| Garcia E., Casali M., 2011, The Sillassen N. B., et al., 2022, A&A, 665, L7 Silverman J. D., et al., 2010, ApJS, 191, 124                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| Simpson J. M., et al., 2014, ApJ, 788, 125                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| L5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| Keel W. C., Motohara K.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| 144, 9                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| M., Brammer G.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| Messenger,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| C., 2017, MNRAS,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| Sobral D., Smail I., Best P. N., Geach J. E., Matsuda Y., Stott J. P.,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| 2012, Cirasuolo                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |

| Sobral D., et al., 2015, MNRAS, 451, 2303 Speagle J. S., Steinhardt C. L., Capak P. L., Silverman J. D., 2014, ApJS, 214, 15                        |
|-----------------------------------------------------------------------------------------------------------------------------------------------------|
| Stach S. M., et al., 2019, MNRAS, 487, 4648                                                                                                         |
| Stach S. M., et al., 2021, MNRAS, 504, 172                                                                                                          |
| Staniszewski Z., et al., 2009, ApJ, 701, 32                                                                                                         |
| Steidel C. C., Adelberger K. L., Dickinson M., Giavalisco M., Pettini M., Kellogg M., 1998, ApJ, 492, 428                                           |
| Steidel C. C., Adelberger K. L., Shapley A. E., Pettini M., Dickinson M., Giavalisco M., 2000, ApJ, 532, 170                                        |
| Steidel C. C., Adelberger K. L., Shapley A. E., Erb D. K., Reddy N. A., Pettini M., 2005, ApJ, 626, 44                                              |
| Stevans M. L., et al., 2021, ApJ, 921, 58                                                                                                           |
| Stott J. P., et al., 2020, MNRAS, 497, 3083                                                                                                         |
| Sunyaev R. A., Zeldovich Y. B., 1972, Comments on Astrophysics and Space Physics, 4, 173                                                            |
| Swinbank A. M., et al., 2014, MNRAS, 438, 1267                                                                                                      |
| Tacconi L. J., et al., 2006, ApJ, 640, 228                                                                                                          |
| Tamura Y., et al., 2009, Nature, 459, 61                                                                                                            |
| Tanaka I., et al., 2011, PASJ, 63, 415                                                                                                              |
| Taylor M. B., 2005, in Shopbell P., Britton M., Society of the Pacific Conference Series Vol.                                                       |
| Ebert R., eds, Astronomical 347, Astronomical Data Analysis Software and Systems XIV. p. 29                                                         |
| Taylor E. N., et al., 2009, ApJS, 183, 295 Tinker J., Kravtsov A. V., Klypin A., Abazajian K., Warren Gottlöber S., Holz D. E., 2008, ApJ, 688, 709 |
| M., Yepes G., Toft S., et al., 2014, ApJ, 782, 68 Treister E., et al., 2009, ApJ, 693, 1713                                                         |
| Trümper J., 1993, Science, 260, 1769                                                                                                                |
| Umehata H., et al., 2015, ApJ, 815, L8                                                                                                              |
| Umehata H., et al., 2019, Science, 366, Venemans B. P., et al., 2002, ApJ, 569,                                                                     |
| 97                                                                                                                                                  |
| Venemans B. P., et al., 2005, A&A, 431,                                                                                                             |
| 793 Virtanen P., et al., 2020, Nature Methods, 17,                                                                                                  |
| 261 Wardlow J. L., et al., 2011, MNRAS, 415, 1479                                                                                                   |
| Weiß A., et al., 2009, ApJ, 707, 1201                                                                                                               |
| Wilkinson A., et al., 2017, MNRAS, 464,                                                                                                             |
| 1380 Williamson R., et al., 2011, ApJ, 738, 139                                                                                                     |
| Wilson G., et al., 2009, ApJ, 698, 1943 Wylezalek D., et al., 2013, ApJ, 769, 79                                                                    |
| Yajima H., et al., 2022, MNRAS, 509, 4037                                                                                                           |
| Yang Y., Zabludoff A., Eisenstein D., Davé 2010, ApJ, 719, 1654                                                                                     |
| Yuan T., et al., 2014, ApJ, 795, L20 Zavala J. A., et al., 2019, ApJ, 887, 183 Zheng X. Z., Cai Z., An F. X., Fan X., Shi                           |
| D. D., MNRAS, 500, 4354 da Cunha E., Charlot S., Elbaz D., 2008,                                                                                    |
| 2021, MNRAS, 1595                                                                                                                                   |
| 388, da Cunha E., et al., 2015, ApJ, 806, 110                                                                                                       |
| da Cunha E., et al., 2021, ApJ, 919, 30                                                                                                             |
| van der Burg R. F. J., Muzzin A.,                                                                                                                   |
| Hoekstra H., Wilson G., Lidman C., H. K. C., 2014, A&A, 561, A79                                                                                    |
| Yee                                                                                                                                                 |
| R.,                                                                                                                                                 |
| L11                                                                                                                                                 |

This paper has been typeset from a T E X/L A T E X file prepared by the author.
Thomas M. Cornish Julie Wardlow Heather Wade David Sobral W. N. Brandt Pierre Cox Helmut Dannerbauer Roberto Decarli Bitten Gullberg Kirsten Knudsen John Stott Mark Swinbank Fabian Walter Paul van der Werf
An ALMA survey of submillimetre galaxies in the Extended Chandra Deep Field South: an unbiased study of SMG environments measured with narrowband imaging
Submillimetre galaxies (SMGs) are some of the most extreme star-forming
systems in the Universe, whose place in the framework of galaxy evolution is as
yet uncertain. It has been hypothesised that SMGs are progenitors of local
early-type galaxies, requiring that SMGs generally reside in galaxy cluster
progenitors at high redshift. We test this hypothesis and explore SMG
environments using a narrowband VLT/HAWK-I+GRAAL study of H$\alpha$ and [OIII]
emitters around an unbiased sample of three ALMA-identified and
spectroscopically-confirmed SMGs at $z \sim 2.3$ and $z \sim 3.3$, where these
SMGs were selected solely on spectroscopic redshift. Comparing with blank-field
observations at similar epochs, we find that one of the three SMGs lies in an
overdensity of emission-line sources on the $\sim4$ Mpc scale of the HAWK-I
field of view, with overdensity parameter $\delta_{g} = 2.6^{+1.4}_{-1.2}$. A
second SMG is significantly overdense only on $\lesssim 1.6$ Mpc scales and the
final SMG is consistent with residing in a blank field environment. The total
masses of the two overdensities are estimated to be $\log(M_{h}/{\rm
M}_{\odot}) =$12.1--14.4, leading to present-day masses of $\log(M_{h,z=0}/{\rm
M}_{\odot}) =$12.9--15.9. These results imply that SMGs occupy a range of
environments, from overdense protoclusters or protogroups to the blank field,
suggesting that while some SMGs are strong candidates for the progenitors of
massive elliptical galaxies in clusters, this may not be their only possible
evolutionary pathway.