title: Detecting Beyond the Standard Model Interactions of Solar Neutrinos in Low-Threshold Dark Matter Detectors

abstract: As low-threshold dark matter detectors advance in development, they will become sensitive to recoils from solar neutrinos which opens up the possibility to explore neutrino properties. We predict the enhancement of the event rate of solar neutrino scattering from Beyond the Standard Model interactions in low-threshold DM detectors, with a focus on silicon, germanium, gallium arsenide, xenon, and argon-based detectors. We consider a set of general neutrino interactions, which fall into five categories: the neutrino magnetic moment as well as interactions mediated by four types of mediators (scalar, pseudoscalar, vector, and axial vector), and consider coupling these mediators to either quarks or electrons. Using these predictions, we place constraints on the mass and couplings of each mediator and the neutrino magnetic moment from current low-threshold detectors like SENSEI, Edelweiss, and SuperCDMS, as well as projections relevant for future experiments such as DAMIC-M, Oscura, Darwin, and ARGO. We find that such low-threshold detectors can improve current constraints by up to two orders of magnitude for vector mediators and one order of magnitude for scalar mediators.


title: Large scale quantum chemistry with Tensor Processing Units

abstract: We demonstrate the use of Google's cloud-based Tensor Processing Units (TPUs) to accelerate and scale up conventional (cubic-scaling) density functional theory (DFT) calculations. Utilizing 512 TPU cores, we accomplish the largest such DFT computation to date, with 247848 orbitals, corresponding to a cluster of 10327 water molecules with 103270 electrons, all treated explicitly. Our work thus paves the way towards accessible and systematic use of conventional DFT, free of any system-specific constraints, at unprecedented scales.


title: Introduction to The Dynamic Pickup and Delivery Problem Benchmark -- ICAPS 2021 Competition

abstract: The Dynamic Pickup and Delivery Problem (DPDP) is an essential problem within the logistics domain. So far, research on this problem has mainly focused on using artificial data which fails to reflect the complexity of real-world problems. In this draft, we would like to introduce a new benchmark from real business scenarios as well as a simulator supporting the dynamic evaluation. The benchmark and simulator have been published and successfully supported the ICAPS 2021 Dynamic Pickup and Delivery Problem competition participated by 152 teams.


title: Challenging exclusive top quark pair production at low and high luminosity LHC

abstract: The elastic production of top quark pairs in $pp$ collisions at low and high luminosity regimes is investigated in detail. We extend the study performed in Phys. Rev. D102, 074014 (2020) which has demonstrated that the sum of two semi-exclusive $t\bar{t}$ production modes, namely in photon-Pomeron and Pomeron-Pomeron interactions, can be experimentally measured when the $t\bar{t}$ system decays semi-leptonically, $t\bar{t}\rightarrow jjbl\nu_l\bar{b}$, both forward protons are tagged and a low amount of pile-up is present. In this study we focus on separating individual channels and a special attention is paid to the situation at high-luminosity LHC environment. We observe that the separation of the pomeron-pomeron and photon-photon events is a challenging task, especially at high amounts of pile-up even with an optimistic 10~ps resolution of timing detectors. In contrast, the photon-Pomeron signal is relatively well separable from all backgrounds at low levels of pile-up, allowing us to discover the elastic $t\bar{t}$ production and probe, for the first time, the production of a top quark pair in the photon-Pomeron interactions. The diffractive photoproduction of such a complex system as the $t\bar{t}$ pair hence can be used not only to study diffractive properties of the scattering amplitude but also to search for new physics beyond the Standard Model, and consequently to be a solid part of the physics programme of forward proton detectors at LHC.


title: Accelerated Quality-Diversity through Massive Parallelism

abstract: Quality-Diversity (QD) optimization algorithms are a well-known approach to generate large collections of diverse and high-quality solutions. However, derived from evolutionary computation, QD algorithms are population-based methods which are known to be data-inefficient and requires large amounts of computational resources. This makes QD algorithms slow when used in applications where solution evaluations are computationally costly. A common approach to speed up QD algorithms is to evaluate solutions in parallel, for instance by using physical simulators in robotics. Yet, this approach is limited to several dozen of parallel evaluations as most physics simulators can only be parallelized more with a greater number of CPUs. With recent advances in simulators that run on accelerators, thousands of evaluations can now be performed in parallel on single GPU/TPU. In this paper, we present QDax, an accelerated implementation of MAP-Elites which leverages massive parallelism on accelerators to make QD algorithms more accessible. We show that QD algorithms are ideal candidates to take advantage of progress in hardware acceleration. We demonstrate that QD algorithms can scale with massive parallelism to be run at interactive timescales without any significant effect on the performance. Results across standard optimization functions and four neuroevolution benchmark environments shows that experiment runtimes are reduced by two factors of magnitudes, turning days of computation into minutes. More surprising, we observe that reducing the number of generations by two orders of magnitude, and thus having significantly shorter lineage does not impact the performance of QD algorithms. These results show that QD can now benefit from hardware acceleration, which contributed significantly to the bloom of deep learning.


title: TOI-1759 b: a transiting sub-Neptune around a low mass star characterized with SPIRou and TESS

abstract: We report the detection and characterization of the transiting sub-Neptune TOI-1759 b, using photometric time-series from TESS and near infrared spectropolarimetric data from SPIRou on the CFHT. TOI-1759 b orbits a moderately active M0V star with an orbital period of $18.849975\pm0.000006$ d, and we measure a planetary radius and mass of $3.06\pm0.22$ R$_\oplus$ and $6.8\pm2.0$ M$_\oplus$. Radial velocities were extracted from the SPIRou spectra using both the CCF and the LBL methods, optimizing the velocity measurements in the near infrared domain. We analyzed the broadband SED of the star and the high-resolution SPIRou spectra to constrain the stellar parameters and thus improve the accuracy of the derived planet parameters. A LSD analysis of the SPIRou Stokes $V$ polarized spectra detects Zeeman signatures in TOI-1759. We model the rotational modulation of the magnetic stellar activity using a GP regression with a quasi-periodic covariance function, and find a rotation period of $35.65^{+0.17}_{-0.15}$ d. We reconstruct the large-scale surface magnetic field of the star using ZDI, which gives a predominantly poloidal field with a mean strength of $18\pm4$ G. Finally, we perform a joint Bayesian MCMC analysis of the TESS photometry and SPIRou RVs to optimally constrain the system parameters. At $0.1176\pm0.0013$ au from the star, the planet receives $6.4$ times the bolometric flux incident on Earth, and its equilibrium temperature is estimated at $433\pm14$ K. TOI-1759 b is a likely gas-dominated sub-Neptune with an expected high rate of photoevaporation. Therefore, it is an interesting target to search for neutral hydrogen escape, which may provide important constraints on the planetary formation mechanisms responsible for the observed sub-Neptune radius desert.


title: Shannon Bounds on Lossy Gray-Wyner Networks

abstract: The Gray-Wyner network subject to a fidelity criterion is studied. Upper and lower bounds for the trade-offs between the private sum-rate and the common rate are obtained for arbitrary sources subject to mean-squared error distortion. The bounds meet exactly, leading to the computation of the rate region, when the source is jointly Gaussian. They meet partially when the sources are modeled via an additive Gaussian "channel". The bounds are inspired from the Shannon bounds on the rate-distortion problem.


title: Efficient Memory Partitioning in Software Defined Hardware

abstract: As programmers turn to software-defined hardware (SDH) to maintain a high level of productivity while programming hardware to run complex algorithms, heavy-lifting must be done by the compiler to automatically partition on-chip arrays. In this paper, we introduce an automatic memory partitioning system that can quickly compute more efficient partitioning schemes than prior systems. Our system employs a variety of resource-saving optimizations and an ML cost model to select the best partitioning scheme from an array of candidates. We compared our system against various state-of-the-art SDH compilers and FPGAs on a variety of benchmarks and found that our system generates solutions that, on average, consume 40.3% fewer logic resources, 78.3% fewer FFs, 54.9% fewer Block RAMs (BRAMs), and 100% fewer DSPs.


title: A semi-discrete numerical scheme for nonlocally regularized KdV-type equations

abstract: A general class of KdV-type wave equations regularized with a convolution-type nonlocality in space is considered. The class differs from the class of the nonlinear nonlocal unidirectional wave equations previously studied by the addition of a linear convolution term involving third-order derivative. To solve the Cauchy problem we propose a semi-discrete numerical method based on a uniform spatial discretization, that is an extension of a previously published work of the present authors. We prove uniform convergence of the numerical method as the mesh size goes to zero. We also prove that the localization error resulting from localization to a finite domain is significantly less than a given threshold if the finite domain is large enough. To illustrate the theoretical results, some numerical experiments are carried out for the Rosenau-KdV equation, the Rosenau-BBM-KdV equation and a convolution-type integro-differential equation. The experiments conducted for three particular choices of the kernel function confirm the error estimates that we provide.


title: NoisyMix: Boosting Model Robustness to Common Corruptions

abstract: For many real-world applications, obtaining stable and robust statistical performance is more important than simply achieving state-of-the-art predictive test accuracy, and thus robustness of neural networks is an increasingly important topic. Relatedly, data augmentation schemes have been shown to improve robustness with respect to input perturbations and domain shifts. Motivated by this, we introduce NoisyMix, a novel training scheme that promotes stability as well as leverages noisy augmentations in input and feature space to improve both model robustness and in-domain accuracy. NoisyMix produces models that are consistently more robust and that provide well-calibrated estimates of class membership probabilities. We demonstrate the benefits of NoisyMix on a range of benchmark datasets, including ImageNet-C, ImageNet-R, and ImageNet-P. Moreover, we provide theory to understand implicit regularization and robustness of NoisyMix.


title: Fixed Points in Supersymmetric Extensions of the Standard Model

abstract: We search for weakly interacting fixed points in extensions of the minimally supersymmetric standard model (MSSM). Necessary conditions lead to three distinct classes of anomaly-free extensions involving either new quark singlets, new quark doublets, or a fourth generation. While interacting fixed points arise prolifically in asymptotically free theories, their existence is significantly constrained as soon as some of the non-abelian gauge sectors are infrared free. Performing a scan over $\sim 200$k different MSSM extensions using matter field multiplicities and the number of superpotential couplings as free parameters, we find mostly infrared conformal fixed points, and a small subset with ultraviolet ones. All settings predict low-scale supersymmetry-breaking and a violation of $R$-parity. Despite of residual interactions, the running of couplings out of asymptotically safe fixed points is logarithmic as in asymptotic freedom. Some fixed points can be matched to the Standard Model though the matching scale comes out too low. Prospects for higher matching scales and asymptotic safety beyond the MSSM are indicated.


title: Automated processing of X-ray computed tomography images via panoptic segmentation for modeling woven composite textiles

abstract: A new, machine learning-based approach for automatically generating 3D digital geometries of woven composite textiles is proposed to overcome the limitations of existing analytical descriptions and segmentation methods. In this approach, panoptic segmentation is leveraged to produce instance segmented semantic masks from X-ray computed tomography (CT) images. This effort represents the first deep learning based automated process for segmenting unique yarn instances in a woven composite textile. Furthermore, it improves on existing methods by providing instance-level segmentation on low contrast CT datasets. Frame-to-frame instance tracking is accomplished via an intersection-over-union (IoU) approach adopted from video panoptic segmentation for assembling a 3D geometric model. A corrective recognition algorithm is developed to improve the recognition quality (RQ). The panoptic quality (PQ) metric is adopted to provide a new universal evaluation metric for reconstructed woven composite textiles. It is found that the panoptic segmentation network generalizes well to new CT images that are similar to the training set but does not extrapolate well to CT images of differing geometry, texture, and contrast. The utility of this approach is demonstrated by capturing yarn flow directions, contact regions between individual yarns, and the spatially varying cross-sectional areas of the yarns.


title: Conciseness of compact $R$-analytic groups

abstract: We prove that every word is strongly concise in the class of compact $R$-analytic groups.


title: FedSpace: An Efficient Federated Learning Framework at Satellites and Ground Stations

abstract: Large-scale deployments of low Earth orbit (LEO) satellites collect massive amount of Earth imageries and sensor data, which can empower machine learning (ML) to address global challenges such as real-time disaster navigation and mitigation. However, it is often infeasible to download all the high-resolution images and train these ML models on the ground because of limited downlink bandwidth, sparse connectivity, and regularization constraints on the imagery resolution. To address these challenges, we leverage Federated Learning (FL), where ground stations and satellites collaboratively train a global ML model without sharing the captured images on the satellites. We show fundamental challenges in applying existing FL algorithms among satellites and ground stations, and we formulate an optimization problem which captures a unique trade-off between staleness and idleness. We propose a novel FL framework, named FedSpace, which dynamically schedules model aggregation based on the deterministic and time-varying connectivity according to satellite orbits. Extensive numerical evaluations based on real-world satellite images and satellite networks show that FedSpace reduces the training time by 1.7 days (38.6%) over the state-of-the-art FL algorithms.


title: DASHA: Distributed Nonconvex Optimization with Communication Compression, Optimal Oracle Complexity, and No Client Synchronization

abstract: We develop and analyze DASHA: a new family of methods for nonconvex distributed optimization problems. When the local functions at the nodes have a finite-sum or an expectation form, our new methods, DASHA-PAGE and DASHA-SYNC-MVR, improve the theoretical oracle and communication complexity of the previous state-of-the-art method MARINA by Gorbunov et al. (2020). In particular, to achieve an epsilon-stationary point, and considering the random sparsifier RandK as an example, our methods compute the optimal number of gradients $\mathcal{O}\left(\frac{\sqrt{m}}{\varepsilon\sqrt{n}}\right)$ and $\mathcal{O}\left(\frac{\sigma}{\varepsilon^{3/2}n}\right)$ in finite-sum and expectation form cases, respectively, while maintaining the SOTA communication complexity $\mathcal{O}\left(\frac{d}{\varepsilon \sqrt{n}}\right)$. Furthermore, unlike MARINA, the new methods DASHA, DASHA-PAGE and DASHA-MVR send compressed vectors only and never synchronize the nodes, which makes them more practical for federated learning. We extend our results to the case when the functions satisfy the Polyak-Lojasiewicz condition. Finally, our theory is corroborated in practice: we see a significant improvement in experiments with nonconvex classification and training of deep learning models.


title: Robust Estimation for Nonparametric Families via Generative Adversarial Networks

abstract: We provide a general framework for designing Generative Adversarial Networks (GANs) to solve high dimensional robust statistics problems, which aim at estimating unknown parameter of the true distribution given adversarially corrupted samples. Prior work focus on the problem of robust mean and covariance estimation when the true distribution lies in the family of Gaussian distributions or elliptical distributions, and analyze depth or scoring rule based GAN losses for the problem. Our work extend these to robust mean estimation, second moment estimation, and robust linear regression when the true distribution only has bounded Orlicz norms, which includes the broad family of sub-Gaussian, sub-Exponential and bounded moment distributions. We also provide a different set of sufficient conditions for the GAN loss to work: we only require its induced distance function to be a cumulative density function of some light-tailed distribution, which is easily satisfied by neural networks with sigmoid activation. In terms of techniques, our proposed GAN losses can be viewed as a smoothed and generalized Kolmogorov-Smirnov distance, which overcomes the computational intractability of the original Kolmogorov-Smirnov distance used in the prior work.

categories: cs.LG eess.SP math.ST stat.CO stat.ML stat.TH


title: High-resolution far-infrared synchrotron FTIR spectroscopy and analysis of the $\nu_7$, $\nu_{19}$ and $\nu_{20}$ bands of trioxane

abstract: Rovibrational bands spectra of three $\nu_{20}$, $\nu_7$ and $\nu_{19}$ bands of 1, 3, 5 -- trioxane (H$_2$CO)$_3$ were recorded in the 50--650 cm$^{-1}$ range using a long path absorption cell coupled to a high resolution Fourier transform spectrometer and synchrotron radiation at the AILES beamline of the SOLEIL synchrotron. More than 16000 lines were assigned with a dRMS better than $0.17\times10^{-3}$ cm$^{-1}$. Two different formalisms (tensorial and Watson) were used to derive accurate rotational and quartic parameters for the three bands and for the first time, a precise determination of Coriolis parameter and $q_+$ $l-$doubling constant for both $\nu_{20}$ and $\nu_{19}$ perpendicular bands was obtained. Last, each set of spectroscopic parameters is compared and discussed between both formalisms.


title: The Drinfel'd centres of String 2-groups

abstract: Let $G$ be a compact connected Lie group and $k \in H^4(BG,\mathbb{Z})$ a cohomology class. The String 2-group $G_k$ is the central extension of $G$ by the 2-group $[\ast/U(1)]$ classified by $k$. It has a close relationship to the level $k$ extension of the loop group $LG$. We compute the Drinfel'd centre of $G_k$ as a smooth 2-group. When $G$ is semisimple, we prove that the Drinfel'd centre is equal to the invertible part of the category of positive energy representations of $LG$ at level $k$ (as long as we exclude factors of $E_8$ at level 2).


title: Jamming Resilient Indoor Factory Deployments: Design and Performance Evaluation

abstract: In the framework of 5G-and-beyond Industry 4.0, jamming attacks for denial of service are a rising threat which can severely compromise the system performance. Therefore, in this paper we deal with the problem of jamming detection and mitigation in indoor factory deployments. We design two jamming detectors based on pseudo-random blanking of subcarriers with orthogonal frequency division multiplexing and consider jamming mitigation with frequency hopping and random scheduling of the user equipments. We then evaluate the performance of the system in terms of achievable BLER with ultra-reliable low-latency communications traffic and jamming missed detection probability. Simulations are performed considering a 3rd Generation Partnership Project spatial channel model for the factory floor with a jammer stationed outside the plant trying to disrupt the communication inside the factory. Numerical results show that jamming resiliency increases when using a distributed access point deployment and exploiting channel correlation among antennas for jamming detection, while frequency hopping is helpful in jamming mitigation only for strict BLER requirements.


title: Beyond Images: Label Noise Transition Matrix Estimation for Tasks with Lower-Quality Features

abstract: The label noise transition matrix, denoting the transition probabilities from clean labels to noisy labels, is crucial for designing statistically robust solutions. Existing estimators for noise transition matrices, e.g., using either anchor points or clusterability, focus on computer vision tasks that are relatively easier to obtain high-quality representations. We observe that tasks with lower-quality features fail to meet the anchor-point or clusterability condition, due to the coexistence of both uninformative and informative representations. To handle this issue, we propose a generic and practical information-theoretic approach to down-weight the less informative parts of the lower-quality features. This improvement is crucial to identifying and estimating the label noise transition matrix. The salient technical challenge is to compute the relevant information-theoretical metrics using only noisy labels instead of clean ones. We prove that the celebrated $f$-mutual information measure can often preserve the order when calculated using noisy labels. We then build our transition matrix estimator using this distilled version of features. The necessity and effectiveness of the proposed method are also demonstrated by evaluating the estimation error on a varied set of tabular data and text classification tasks with lower-quality features. Code is available at github.com/UCSC-REAL/BeyondImages.


title: Principled Graph Management

abstract: Graph Generation is a recently introduced enhanced Column Generation algorithm for solving expanded Linear Programming relaxations of mixed integer linear programs without weakening the expanded relaxations which characterize these methods. To apply Graph Generation we must be able to map any given column generated during pricing to a small directed acyclic graph for which any path from source to sink describes a feasible column. This structure is easily satisfied for vehicle routing, crew scheduling and various logistics problems where pricing is a constrained shortest path problem. The construction of such graphs trades off the size/diversity of a subset of columns modeled by the graphs versus the additional computational time required to solve the problems induced by larger graphs. Graph Generation (GG) has two computational bottlenecks. The first is pricing. Pricing in GG and Column Generation (CG) is identical because of the structure of the problems solved. The second bottleneck is the restricted master problem (RMP), which is more computationally intensive in GG than in CG given the same number of columns generated. By design GG converges in fewer iterations than CG, and hence requires fewer calls to pricing. Therefore, when the computation time of GG is dominated by pricing, as opposed to solving the RMP, GG converges much faster than CG in terms of time. However GG need not converge faster than CG when the GG RMP, rather than pricing, dominates computation. In this paper we introduce Principled Graph Management (PGM), which is an algorithm to solve the GG RMP rapidly by exploiting its special structure. We demonstrate the effectiveness of PGM inside a GG solution to the classical Capacitated Vehicle Routing Problem. We demonstrate that PGM solves the GG RMP hundreds of times faster than the baseline solver and that the improvement in speed increases with problem size.


title: Topological Classification in a Wasserstein Distance Based Vector Space

abstract: Classification of large and dense networks based on topology is very difficult due to the computational challenges of extracting meaningful topological features from real-world networks. In this paper we present a computationally tractable approach to topological classification of networks by using principled theory from persistent homology and optimal transport to define a novel vector representation for topological features. The proposed vector space is based on the Wasserstein distance between persistence barcodes. The 1-skeleton of the network graph is employed to obtain 1-dimensional persistence barcodes that represent connected components and cycles. These barcodes and the corresponding Wasserstein distance can be computed very efficiently. The effectiveness of the proposed vector space is demonstrated using support vector machines to classify simulated networks and measured functional brain networks.


title: Maximum likelihood reconstruction of water Cherenkov events with deep generative neural networks

abstract: Large water Cherenkov detectors have shaped our current knowledge of neutrino physics and nucleon decay, and will continue to do so in the foreseeable future. These highly capable detectors allow for directional and topological, as well as calorimetric information to be extracted from signals on their photosensors. The current state-of-the-art approach to water Cherenkov reconstruction relies on maximum-likelihood estimation, with several simplifying assumptions employed to make the problem tractable. In this paper, we describe neural networks that produce probability density functions for the signals at each photosensor, given a set of inputs that characterizes a particle in the detector. The neural networks we propose allow for likelihood-based approaches to event reconstruction with significantly fewer assumptions compared to traditional methods, and are thus expected to improve on the current performance of water Cherenkov detectors.


title: Global Optimization Networks

abstract: We consider the problem of estimating a good maximizer of a black-box function given noisy examples. To solve such problems, we propose to fit a new type of function which we call a global optimization network (GON), defined as any composition of an invertible function and a unimodal function, whose unique global maximizer can be inferred in $\mathcal{O}(D)$ time. In this paper, we show how to construct invertible and unimodal functions by using linear inequality constraints on lattice models. We also extend to \emph{conditional} GONs that find a global maximizer conditioned on specified inputs of other dimensions. Experiments show the GON maximizers are statistically significantly better predictions than those produced by convex fits, GPR, or DNNs, and are more reasonable predictions for real-world problems.


title: Determinantal Formulas for Exceptional Orthogonal Polynomials

abstract: We present determinantal formulas for families of exceptional $X_m$-Laguerre and exceptional $X_m$-Jacobi polynomials and also for exceptional $X_2$-Hermite polynomials. The formulas resemble Vandermonde determinants and use the zeros of the classical orthogonal polynomials.


title: PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts

abstract: PromptSource is a system for creating, sharing, and using natural language prompts. Prompts are functions that map an example from a dataset to a natural language input and target output. Using prompts to train and query language models is an emerging area in NLP that requires new tools that let users develop and refine these prompts collaboratively. PromptSource addresses the emergent challenges in this new setting with (1) a templating language for defining data-linked prompts, (2) an interface that lets users quickly iterate on prompt development by observing outputs of their prompts on many examples, and (3) a community-driven set of guidelines for contributing new prompts to a common pool. Over 2,000 prompts for roughly 170 datasets are already available in PromptSource. PromptSource is available at https://github.com/bigscience-workshop/promptsource.


title: Combinatorial Gray codes-an updated survey

abstract: A combinatorial Gray code for a class of objects is a listing that contains each object from the class exactly once such that any two consecutive objects in the list differ only by a `small change'. Such listings are known for many different combinatorial objects, including bitstrings, combinations, permutations, partitions, triangulations, but also for objects defined with respect to a fixed graph, such as spanning trees, perfect matchings or vertex colorings. This survey provides a comprehensive picture of the state-of-the-art of the research on combinatorial Gray codes. In particular, it gives an update on Savage's influential survey [C. D. Savage. A survey of combinatorial Gray codes. SIAM Rev., 39(4):605-629, 1997.], incorporating many more recent developments. We also elaborate on the connections to closely related problems in graph theory, algebra, order theory, geometry and algorithms, which embeds this research area into a broader context. Lastly, we collect and propose a number of challenging research problems, thus stimulating new research endeavors.


title: An Experience Report of Executive-Level Artificial Intelligence Education in the United Arab Emirates

abstract: Teaching artificial intelligence (AI) is challenging. It is a fast moving field and therefore difficult to keep people updated with the state-of-the-art. Educational offerings for students are ever increasing, beyond university degree programs where AI education traditionally lay. In this paper, we present an experience report of teaching an AI course to business executives in the United Arab Emirates (UAE). Rather than focusing only on theoretical and technical aspects, we developed a course that teaches AI with a view to enabling students to understand how to incorporate it into existing business processes. We present an overview of our course, curriculum and teaching methods, and we discuss our reflections on teaching adult learners, and to students in the UAE.


title: Upper bounds for the moduli of polynomial-like maps

abstract: We establish a version of the Pommerenke-Levin-Yoccoz inequality for the modulus of a polynomial-like restriction of a global polynomial and give two applications. First it is shown that if the modulus of a polynomial-like restriction of an arbitrary polynomial is bounded from below then this forces bounded combinatorics. The second application concerns parameter slices of cubic polynomials given by a non-repelling value of a fixed point multiplier. Namely, the intersection of the main cubioid and the multiplier slice lies in the closure of the principal hyperbolic domain, with only possible exception of queer components.


title: Multivariate nonparametric regression by least squares Jacobi polynomials approximations

abstract: In this work, we study a random orthogonal projection based least squares estimator for the stable solution of a multivariate nonparametric regression (MNPR) problem. More precisely, given an integer $d\geq 1$ corresponding to the dimension of the MNPR problem, a positive integer $N\geq 1$ and a real parameter $\alpha\geq -\frac{1}{2},$ we show that a fairly large class of $d-$variate regression functions are well and stably approximated by its random projection over the orthonormal set of tensor product $d-$variate Jacobi polynomials with parameters $(\alpha,\alpha).$ The associated uni-variate Jacobi polynomials have degree at most $N$ and their tensor products are orthonormal over $\mathcal U=[0,1]^d,$ with respect to the associated multivariate Jacobi weights. In particular, if we consider $n$ random sampling points $\mathbf X_i$ following the $d-$variate Beta distribution, with parameters $(\alpha+1,\alpha+1),$ then we give a relation involving $n, N, \alpha$ to ensure that the resulting $(N+1)^d\times (N+1)^d$ random projection matrix is well conditioned. Moreover, we provide squared integrated as well as $L^2-$risk errors of this estimator. Precise estimates of these errors are given in the case where the regression function belongs to an isotropic Sobolev space $H^s(I^d),$ with $s> \frac{d}{2}.$ Also, to handle the general and practical case of an unknown distribution of the $\mathbf X_i,$ we use Shepard's scattered interpolation scheme in order to generate fairly precise approximations of the observed data at $n$ i.i.d. sampling points $\mathbf X_i$ following a $d-$variate Beta distribution. Finally, we illustrate the performance of our proposed multivariate nonparametric estimator by some numerical simulations with synthetic as well as real data.


title: Dr.Jit: A Just-In-Time Compiler for Differentiable Rendering

abstract: Dr.Jit is a new just-in-time compiler for physically based rendering and its derivative. Dr.Jit expedites research on these topics in two ways: first, it traces high-level simulation code (e.g., written in Python) and aggressively simplifies and specializes the resulting program representation, producing data-parallel kernels with state-of-the-art performance on CPUs and GPUs. Second, it simplifies the development of differentiable rendering algorithms. Efficient methods in this area turn the derivative of a simulation into a simulation of the derivative. Dr.Jit provides fine-grained control over the process of automatic differentiation to help with this transformation. Specialization is particularly helpful in the context of differentiation, since large parts of the simulation ultimately do not influence the computed gradients. Dr.Jit tracks data dependencies globally to find and remove redundant computation.


title: Mott versus hybridization gap in the low-temperature phase of $1T$-TaS$_2$

abstract: We compute the correlated electronic structure of stacked $1T$-TaS$_2$ bilayers using the $GW$ + EDMFT method. Depending on the surface termination, the semi-infinite uncorrelated system is either band-insulating or exhibits a metallic surface state. For realistic values of the onsite and intersite interactions, a Mott gap opens in the surface state, but this gap is smaller than the gap originating from the bilayer structure. Our results are consistent with recent scanning tunneling spectroscopy measurements for different terminating layers, and with our own photoemission measurements, which indicate the coexistence of spatial regions with different gaps in the electronic spectrum.


title: ASR-Aware End-to-end Neural Diarization

abstract: We present a Conformer-based end-to-end neural diarization (EEND) model that uses both acoustic input and features derived from an automatic speech recognition (ASR) model. Two categories of features are explored: features derived directly from ASR output (phones, position-in-word and word boundaries) and features derived from a lexical speaker change detection model, trained by fine-tuning a pretrained BERT model on the ASR output. Three modifications to the Conformer-based EEND architecture are proposed to incorporate the features. First, ASR features are concatenated with acoustic features. Second, we propose a new attention mechanism called contextualized self-attention that utilizes ASR features to build robust speaker representations. Finally, multi-task learning is used to train the model to minimize classification loss for the ASR features along with diarization loss. Experiments on the two-speaker English conversations of Switchboard+SRE data sets show that multi-task learning with position-in-word information is the most effective way of utilizing ASR features, reducing the diarization error rate (DER) by 20% relative to the baseline.


title: Fenrir: Physics-Enhanced Regression for Initial Value Problems

abstract: We show how probabilistic numerics can be used to convert an initial value problem into a Gauss--Markov process parametrised by the dynamics of the initial value problem. Consequently, the often difficult problem of parameter estimation in ordinary differential equations is reduced to hyperparameter estimation in Gauss--Markov regression, which tends to be considerably easier. The method's relation and benefits in comparison to classical numerical integration and gradient matching approaches is elucidated. In particular, the method can, in contrast to gradient matching, handle partial observations, and has certain routes for escaping local optima not available to classical numerical integration. Experimental results demonstrate that the method is on par or moderately better than competing approaches.


title: Imitation Learning by Estimating Expertise of Demonstrators

abstract: Many existing imitation learning datasets are collected from multiple demonstrators, each with different expertise at different parts of the environment. Yet, standard imitation learning algorithms typically treat all demonstrators as homogeneous, regardless of their expertise, absorbing the weaknesses of any suboptimal demonstrators. In this work, we show that unsupervised learning over demonstrator expertise can lead to a consistent boost in the performance of imitation learning algorithms. We develop and optimize a joint model over a learned policy and expertise levels of the demonstrators. This enables our model to learn from the optimal behavior and filter out the suboptimal behavior of each demonstrator. Our model learns a single policy that can outperform even the best demonstrator, and can be used to estimate the expertise of any demonstrator at any state. We illustrate our findings on real-robotic continuous control tasks from Robomimic and discrete environments such as MiniGrid and chess, out-performing competing methods in $21$ out of $23$ settings, with an average of $7\%$ and up to $60\%$ improvement in terms of the final reward.


title: Systems Mining with Heraklit: The Next Step

abstract: We suggest systems mining as the next step after process mining. Systems mining starts with a more careful investigation of runs, and constructs a detailed model of behavior, more subtle than classical process mining. The resulting model is enriched with information about data. From this model, a system model can be deduced in a systematic way.


title: Cyclical Pruning for Sparse Neural Networks

abstract: Current methods for pruning neural network weights iteratively apply magnitude-based pruning on the model weights and re-train the resulting model to recover lost accuracy. In this work, we show that such strategies do not allow for the recovery of erroneously pruned weights. To enable weight recovery, we propose a simple strategy called \textit{cyclical pruning} which requires the pruning schedule to be periodic and allows for weights pruned erroneously in one cycle to recover in subsequent ones. Experimental results on both linear models and large-scale deep neural networks show that cyclical pruning outperforms existing pruning algorithms, especially at high sparsity ratios. Our approach is easy to tune and can be readily incorporated into existing pruning pipelines to boost performance.


title: Computer sciences and synthesis: retrospective and perspective

abstract: The problem of synthesis in computer sciences, including cybernetics, artificial intelligence and system analysis, is analyzed. Main methods of realization this problem are discussed. Ways of search universal method of creation universal synthetic science are represented. As example of such universal method polymetric analysis is given. Perspective of further development of this research, including application polymetric method for the resolution main problems of computer sciences, is analyzed too.


title: Improved Regret for Differentially Private Exploration in Linear MDP

abstract: We study privacy-preserving exploration in sequential decision-making for environments that rely on sensitive data such as medical records. In particular, we focus on solving the problem of reinforcement learning (RL) subject to the constraint of (joint) differential privacy in the linear MDP setting, where both dynamics and rewards are given by linear functions. Prior work on this problem due to Luyo et al. (2021) achieves a regret rate that has a dependence of $O(K^{3/5})$ on the number of episodes $K$. We provide a private algorithm with an improved regret rate with an optimal dependence of $O(\sqrt{K})$ on the number of episodes. The key recipe for our stronger regret guarantee is the adaptivity in the policy update schedule, in which an update only occurs when sufficient changes in the data are detected. As a result, our algorithm benefits from low switching cost and only performs $O(\log(K))$ updates, which greatly reduces the amount of privacy noise. Finally, in the most prevalent privacy regimes where the privacy parameter $\epsilon$ is a constant, our algorithm incurs negligible privacy cost -- in comparison with the existing non-private regret bounds, the additional regret due to privacy appears in lower-order terms.


title: Orthogonal Fold & Cut

abstract: We characterize the cut patterns that can be produced by "orthogonal fold & cut": folding an axis-aligned rectangular sheet of paper along horizontal and vertical creases, and then making a single straight cut (at any angle). Along the way, we solve a handful of related problems: orthogonal fold & punch, 1D fold & cut, signed 1D fold & cut, and 1D interval fold & cut.


title: Relativistic coupled cluster calculations of the electron affinity and ionization potential of Nh(113)

abstract: Theoretical calculations based on the Dirac--Coulomb--Breit relativistic coupled cluster method have been carried out for the electron affinities and ionization potentials of the superheavy element nihonium (Nh) and its lighter homologues In and Tl. The In and Tl calculations are in agreement with measurement within uncertainties. For Nh, where experiment is yet unknown, we predict the ionization potential of 7.569(48) eV and electron affinity of 0.776(30) eV.


title: Singular metrics and a conjecture by Campana and Peternell

abstract: A conjecture by Campana and Peternell says that if a positive multiple of $K_X$ is linearly equivalent to an effective divisor $D$ plus a pseudo-effective divisor, then the Kodaira dimension of $X$ should be at least as big as the Iitaka dimension of $D$. This is a very useful generalization of the non-vanishing conjecture (which is the case $D = 0$). We use recent work about singular metrics on pluri-adjoint bundles to show that the Campana-Peternell conjecture is almost equivalent to the non-vanishing conjecture.


title: Sidon sets in a union of intervals

abstract: We study the maximum size of Sidon sets in unions of integers intervals. If $A\subseteq\mathbb{N}$ is the union of two intervals and if $\left| A \right|=n$ (where $\left| A \right|$ denotes the cardinality of $A$), we prove that $A$ contains a Sidon set of size at least $0, 876\sqrt{n}$. On the other hand, by using the small differences technique, we establish a bound of the maximum size of Sidon sets in the union of $k$ intervals.


title: Age Distribution in Arbitrary Preemptive Memoryless Networks

abstract: We study the probability distribution of age of information (AoI) in arbitrary networks with memoryless service times. A source node generates packets following a Poisson process, and then the packets are forwarded across the network in such a way that newer updates preempt older ones. This model is equivalent to gossip networks that was recently studied by Yates, and for which he obtained a recursive formula allowing the computation for the average AoI. In this paper, we obtain a very simple characterization of the stationary distribution of AoI at every node in the network. This allows for the computation of the average of an arbitrary function of the age. In particular, we can compute age-violation probabilities. Furthermore, we show how it is possible to use insights from our simple characterization in order to substantially reduce the computation time of average AoIs in some structured networks. Finally, we describe how it is possible to use our characterization in order to obtain faster and more accurate Monte Carlo simulations estimating the average AoI, or the average of an arbitrary function of the age.


title: Free noncommutative hereditary kernels: Jordan decomposition, Arveson extension, kernel domination

abstract: We discuss a (i) quantized version of the Jordan decomposition theorem for a complex Borel measure on a compact Hausdorff space, namely, the more general problem of decomposing a general noncommutative kernel (a quantization of the standard notion of kernel function) as a linear combination of completely positive noncommutative kernels (a quantization of the standard notion of positive definite kernel). Other special cases of (i) include: the problem of decomposing a general operator-valued kernel function as a linear combination of positive kernels (not always possible), of decomposing a general bounded linear Hilbert-space operator as a linear combination of positive linear operators (always possible), of decomposing a completely bounded linear map from a $C^*$-algebra ${\mathcal A}$ to an injective $C^*$-algebra ${\mathcal L}({\mathcal Y})$ as a linear combination of completely positive maps from ${\mathcal A}$ to ${\mathcal L}({\mathcal Y})$ (always possible). We also discuss (ii) a noncommutative kernel generalization of the Arveson extension theorem (any completely positive map $\phi$ from a operator system ${\mathbb S}$ to an injective $C^*$-algebra ${\mathcal L}({\mathcal Y})$ can be extended to a completely positive map $\phi_e$ from a $C^*$-algebra containing ${\mathbb S}$ to ${\mathcal L}({\mathcal Y})$), and (iii) a noncommutative kernel version of a Positivstellensatz (i.e., finding a certificate to explain why one kernel is positive at points where another given kernel is positive).


title: On Coded Caching Systems with Offline Users

abstract: Coded caching is a technique that leverages locally cached contents at the users to reduce the network's peak-time communication load. Coded caching achieves significant performance gains compared to uncoded caching schemes and is thus a promising technique to boost performance in future networks. In the original model introduced by Maddah-Ali and Niesen (MAN), a server stores multiple files and is connected to multiple cache-aided users through an error-free shared link; once the local caches have been filled and all users have sent their demand to the server, the server can start sending coded multicast messages to satisfy all users' demands. A practical limitation of the original MAN model is that it halts if the server does not receive all users' demands, which is the limiting case of asynchronous coded caching when the requests of some users arrive with infinite delay. In this paper we formally define a coded caching system where some users are offline. We propose achievable and converse bounds for this novel setting and show under which conditions they meet, thus providing an optimal solution, and when they are to within a constant multiplicative gap of two. Interestingly, when optimality can be be shown, the optimal load-memory tradeoff only depends on the number active users, and not on the total (active plus offline) number of users.


title: Causal Inference Through the Structural Causal Marginal Problem

abstract: We introduce an approach to counterfactual inference based on merging information from multiple datasets. We consider a causal reformulation of the statistical marginal problem: given a collection of marginal structural causal models (SCMs) over distinct but overlapping sets of variables, determine the set of joint SCMs that are counterfactually consistent with the marginal ones. We formalise this approach for categorical SCMs using the response function formulation and show that it reduces the space of allowed marginal and joint SCMs. Our work thus highlights a new mode of falsifiability through additional variables, in contrast to the statistical one via additional data.


title: Common reducing subspaces and decompositions of contractions

abstract: A commuting triple of Hilbert space operators $(A,B,P)$, for which the closed tetrablock $\overline{\mathbb E}$ is a spectral set, is called a \textit{tetrablock-contraction} or simply an $\mathbb E$-\textit{contraction}, where \[ \mathbb E=\{(x_1,x_2,x_3)\in \mathbb C^3:\, 1-x_1z-x_2w+x_3zw \neq 0 \quad \text{ whenever } \; |z|\leq 1, \; \; |w|\leq 1 \} \subset \mathbb C^3, \] is a polynomially convex domain which is naturally associated with the $\mu$-synthesis problem. By applications of the theory of $\mathbb E$-contractions, we obtain several results on decompositions of contractions.


title: A Comparison of Online Hate on Reddit and 4chan: A Case Study of the 2020 US Election

abstract: The rapid integration of the Internet into our daily lives has led to many benefits but also to a number of new, wide-spread threats such as online hate, trolling, bullying, and generally aggressive behaviours. While research has traditionally explored online hate, in particular, on one platform, the reality is that such hate is a phenomenon that often makes use of multiple online networks. In this article, we seek to advance the discussion into online hate by harnessing a comparative approach, where we make use of various Natural Language Processing (NLP) techniques to computationally analyse hateful content from Reddit and 4chan relating to the 2020 US Presidential Elections. Our findings show how content and posting activity can differ depending on the platform being used. Through this, we provide initial comparison into the platform-specific behaviours of online hate, and how different platforms can serve specific purposes. We further provide several avenues for future research utilising a cross-platform approach so as to gain a more comprehensive understanding of the global hate ecosystem.


title: Electric field screening in pair discharges and generation of pulsar radio emission

abstract: Pulsar radio emission may be generated in pair discharges which fill the pulsar magnetosphere with plasma as an accelerating electric field is screened by freshly created pairs. In this Letter we develop a simplified analytic theory for the screening of the electric field in these pair discharges and use it to estimate total radio luminosity and spectrum. The discharge has three stages. First, the electric field is screened for the first time and starts to oscillate. Next, a nonlinear phase occurs. In this phase, the amplitude of the electric field experiences strong damping because the field dramatically changes the momenta of newly created pairs. This strong damping ceases, and the system enters a final linear phase, when the electric field can no longer dramatically change pair momenta. Applied to pulsars, this theory may explain several aspects of radio emission, including the observed luminosity, $L_{\rm{rad}} \sim 10^{28} \rm{erg} \, \rm{s}^{-1}$, and the observed spectrum, $S_\omega \sim \omega^{-1.4 \pm 1.0} $.


title: Theorems motivated by foundations of quantum mechanics and some of their applications

abstract: This paper provides theorems aimed at shedding light on issues in the foundations of quantum mechanics. These theorems can be used to propose new interpretations to the theory, or to better understand, evaluate and improve current interpretations. Some of these applications include: (1) A proof of the existence of pilot-wave theories that are fully equivalent to standard quantum mechanics in a path-wise sense. This equivalence is stronger than what is entailed from the more traditional requirements of equivariance, or good mixing properties, and is necessary to assure proper correlations across time and proper records of the past. (2) A proposal for a minimalistic ontology for non-collapse quantum mechanics, in which Born's rule provides the proper predictions.(3) The observation of a close relationship between Born's rule and a version of the superposition principle.


title: Hecke algebras for $p$-adic reductive groups and Local Langlands Correspondence for Bernstein blocks

abstract: We study the endomorphism algebras attached to Bernstein components of reductive $p$-adic groups and construct a local Langlands correspondence with the appropriate set of enhanced $L$-parameters, using certain "desiderata" properties for the LLC for supercuspidal representations of proper Levi subgroups. We give several applications of our LLC to various reductive groups with Bernstein blocks cuspidally supported on general linear groups. In particular, for Levi subgroups of maximal parabolic of the split exceptional group $G_2$, we compute the explicit weight functions for the corresponding Hecke algebras, and show that they satisfy a conjecture of Lusztig's. Some results from $\S 4$ are used by the same authors to construct a full local Langlands correspondence in \cite{AX-LLC}. Moreover, we also prove a reduction to depth zero case result for the Bernstein components attached to regular supercuspidal representations of Levi subgroups.


title: Harmony: Overcoming the Hurdles of GPU Memory Capacity to Train Massive DNN Models on Commodity Servers

abstract: Deep neural networks (DNNs) have grown exponentially in size over the past decade, leaving only those who have massive datacenter-based resources with the ability to develop and train such models. One of the main challenges for the long tail of researchers who might have only limited resources (e.g., a single multi-GPU server) is limited GPU memory capacity compared to model size. The problem is so acute that the memory requirement of training massive DNN models can often exceed the aggregate capacity of all available GPUs on a single server; this problem only gets worse with the trend of ever-growing model sizes. Current solutions that rely on virtualizing GPU memory (by swapping to/from CPU memory) incur excessive swapping overhead. In this paper, we present a new training framework, Harmony, and advocate rethinking how DNN frameworks schedule computation and move data to push the boundaries of training massive models efficiently on a single commodity server. Across various massive DNN models, Harmony is able to reduce swap load by up to two orders of magnitude and obtain a training throughput speedup of up to 7.6x over highly optimized baselines with virtualized memory.


title: On the phase-mixed eccentricity and inclination distributions of wide binaries in the Galaxy

abstract: Modern observational surveys allow us to probe the phase space distribution function (DF) of wide binaries in the Solar neighbourhood. This DF exhibits non-trivial features, in particular a superthermal distribution of eccentricities for semimajor axes $a\gtrsim 10^3$AU. To interpret such features we must first understand how the binary DF is affected by dynamical perturbations, which typically fall into two classes: (i) stochastic kicks from passing stars, molecular clouds, etc. and (ii) secular torques from the Galactic tide. Here we isolate effect (ii) and calculate the time-asymptotic, phase-mixed DF for an ensemble of wide binaries under quadrupole-order tides. For binaries wide enough that the phase-mixing assumption is valid, none of our results depend explicitly on semimajor axes, masses, etc. We show that unless the initial DF is both isotropic in binary orientation and thermal in eccentricity, then the final phase-mixed DF is always both anisotropic and non-thermal. However, the only way to produce a superthermal DF under phase mixing is for the initial DF to itself be superthermal.


title: Impact Analysis of Harassment Against Women Using Association Rule Mining Approaches: Bangladesh Prospective

abstract: In recent years, it has been noticed that women are making progress in every sector of society. Their involvement in every field, such as education, job market, social work, etc., is increasing at a remarkable rate. For the last several years, the government has been trying its level best for the advancement of women in every sector by doing several research work and activities and funding several organizations to motivate women. Although women's involvement in several fields is increasing, the big concern is they are facing several barriers in their advancement, and it is not surprising that sexual harassment is one of them. In Bangladesh, harassment against women, especially students, is a common phenomenon, and it is increasing. In this paper, a survey-based and Apriori algorithm are used to analyze the several impacts of harassment among several age groups. Also, several factors such as frequent impacts of harassment, most vulnerable groups, women mostly facing harassment, the alleged person behind harassment, etc., are analyzed through association rule mining of Apriori algorithm and F.P. Growth algorithm. And then, a comparison of performance between both algorithms has been shown briefly. For this analysis, data have been carefully collected from all ages.


title: Multi-Resolution Factor Graph Based Stereo Correspondence Algorithm

abstract: A dense depth-map of a scene at an arbitrary view orientation can be estimated from dense view correspondences among multiple lower-dimensional views of the scene. These low-dimensional view correspondences are dependent on the geometrical relationship among the views and the scene. Determining dense view correspondences is difficult in part due to presence of homogeneous regions in the scene and due to presence of occluded regions and illumination differences among the views. We present a new multi-resolution factor graph-based stereo matching algorithm (MR-FGS) that utilizes both intra- and inter-resolution dependencies among the views as well as among the disparity estimates. The proposed framework allows exchange of information among multiple resolutions of the correspondence problem and is useful for handling larger homogeneous regions in a scene. The MR-FGS algorithm was evaluated qualitatively and quantitatively using stereo pairs in the Middlebury stereo benchmark dataset based on commonly used performance measures. When compared to a recently developed factor graph model (FGS), the MR-FGS algorithm provided more accurate disparity estimates without requiring the commonly used post-processing procedure known as the left-right consistency check. The multi-resolution dependency constraint within the factor-graph model significantly improved contrast along depth boundaries in the MR-FGS generated disparity maps.


title: Webs and canonical bases in degree two

abstract: We show that Lusztig's canonical basis for the degree two part of the Grassmannian coordinate ring is given by SL(k) web diagrams. Equivalently, we show that every SL(2) web immanant of a plabic graph for Gr(k,n) is an SL(k) web invariant.


title: Influence of sub-system non-idealities on the performance of Gaussian modulated CV-QKD

abstract: We present a detailed analysis of the numerical modelling and evaluation of sub-systems in a Gaussian modulated CV-QKD system, incorporating non-ideal operations, and along with associated results.


title: Causal Imitation Learning under Temporally Correlated Noise

abstract: We develop algorithms for imitation learning from policy data that was corrupted by temporally correlated noise in expert actions. When noise affects multiple timesteps of recorded data, it can manifest as spurious correlations between states and actions that a learner might latch on to, leading to poor policy performance. To break up these spurious correlations, we apply modern variants of the instrumental variable regression (IVR) technique of econometrics, enabling us to recover the underlying policy without requiring access to an interactive expert. In particular, we present two techniques, one of a generative-modeling flavor (DoubIL) that can utilize access to a simulator, and one of a game-theoretic flavor (ResiduIL) that can be run entirely offline. We find both of our algorithms compare favorably to behavioral cloning on simulated control tasks.


title: On the importance of electron-electron and electron-phonon scatterings and energy renormalizations during carrier relaxation in monolayer transition-metal dichalcogenides

abstract: An $\it{ab \,\, initio}$ based fully microscopic many-body approach is used to study the carrier relaxation dynamics in monolayer transition-metal dichalcogenides. Bandstructures and wavefunctions as well as phonon energies and coupling matrix elements are calculated using density functional theory. The resulting dipole and Coulomb matrix elements are implemented in the Dirac-Bloch equations to calculate carrier-carrier and carrier-phonon scatterings throughout the whole Brillouin zone. It is shown that carrier scatterings lead to a relaxation into hot quasi-Fermi distributions on a single femtosecond timescale. Carrier cool down and inter-valley transitions are mediated by phonon scatterings on a picosecond timescale. Strong, density-dependent energy renormalizations are shown to be valley-dependent. For MoTe$_2$, MoSe$_2$ and MoS$_2$ the change of energies with occupation is found to be about 50$\%$ stronger in the $\Sigma$ and $\Lambda$ side valleys than in the $K$ and $K'$ valleys. However, for realistic carrier densities, the materials always maintain their direct bandgap at the $K$ points of the Brillouin zone.


title: Gradient estimators for normalising flows

abstract: Recently a machine learning approach to Monte-Carlo simulations called Neural Markov Chain Monte-Carlo (NMCMC) is gaining traction. In its most popular form it uses neural networks to construct normalizing flows which are then trained to approximate the desired target distribution. In this contribution we present new gradient estimator for Stochastic Gradient Descent algorithm (and the corresponding \texttt{PyTorch} implementation) and show that it leads to better training results for $\phi^4$ model. For this model our estimator achieves the same precision in approximately half of the time needed in standard approach and ultimately provides better estimates of the free energy. We attribute this effect to the lower variance of the new estimator. In contrary to the standard learning algorithm our approach does not require estimation of the action gradient with respect to the fields, thus has potential of further speeding up the training for models with more complicated actions.

categories: stat.ML cond-mat.stat-mech cs.LG hep-lat


title: Approximating Full Conformal Prediction at Scale via Influence Functions

abstract: Conformal prediction (CP) is a wrapper around traditional machine learning models, giving coverage guarantees under the sole assumption of exchangeability; in classification problems, for a chosen significance level $\varepsilon$, CP guarantees that the error rate is at most $\varepsilon$, irrespective of whether the underlying model is misspecified. However, the prohibitive computational costs of "full" CP led researchers to design scalable alternatives, which alas do not attain the same guarantees or statistical power of full CP. In this paper, we use influence functions to efficiently approximate full CP. We prove that our method is a consistent approximation of full CP, and empirically show that the approximation error becomes smaller as the training set increases; e.g., for $10^{3}$ training points the two methods output p-values that are $<10^{-3}$ apart: a negligible error for any practical application. Our methods enable scaling full CP to large real-world datasets. We compare our full CP approximation (ACP) to mainstream CP alternatives, and observe that our method is computationally competitive whilst enjoying the statistical predictive power of full CP.


title: Free-Energy Transduction in Chemical Reaction Networks: from Enzymes to Metabolism

abstract: We provide a rigorous definition of free-energy transduction and its efficiency in arbitrary -- linear or nonlinear -- open chemical reaction networks (CRNs) operating at steady state. Our method is based on the knowledge of the stoichiometric matrix and of the chemostatted species (i.e. the species maintained at constant concentration by the environment) to identify the fundamental currents and forces contributing to the entropy production. Transduction occurs when the current of a stoichiometrically balanced process is driven against its spontaneous direction (set by its force) thanks to other processes flowing along their spontaneous direction. In these regimes, open CRNs operate as thermodynamic machines. After exemplifying these general ideas using toy models, we analyze central energy metabolism. We relate the fundamental currents to metabolic pathways and discuss the efficiency with which they are able to transduce free energy.


title: Governing online goods: Maturity and formalization in Minecraft, Reddit, and World of Warcraft communities

abstract: Building a successful community means governing active populations and limited resources. This challenge often requires communities to design formal governance systems from scratch. But the characteristics of successful institutional designs are unclear. Communities that are more mature and established may have more elaborate formal policy systems. Alternatively, they may require less formalization precisely because of their maturity. Indeed, scholars often downplay the role that formal rules relative to unwritten rules, norms, and values. But in a community with formal rules, decisions are more consistent, transparent, and legitimate. To understand the relationship of formal institutions to community maturity and governance style, we conduct a large-scale quantitative analysis applying institutional analysis frameworks of self-governance scholar Elinor Ostrom to 80,000 communities across 3 platforms: the sandbox game Minecraft, the MMO game World of Warcraft, and Reddit. We classify communities' written rules to test predictors of institutional formalization. From this analysis we extract two major findings. First, institutional formalization, the size and complexity of an online community's governance system, is generally positively associated with maturity, as measured by age, population size, or degree of user engagement. Second, we find that online communities employ similar governance styles across platforms, strongly favoring "weak" norms to "strong" requirements. These findings suggest that designers and founders of online communities converge on styles of governance practice that are correlated with successful self-governance. With deeper insights into the patterns of successful self-governance, we can help more communities overcome the challenges of self-governance and create for their members powerful experiences of shared meaning and collective empowerment.


title: Finite orbit width effects in large aspect ratio stellarators

abstract: New orbit averaged equations for low collisionality neoclassical fluxes in large aspect ratio stellarators with mirror ratios close to unity are derived. The equations retain finite orbit width effects by employing the second adiabatic invariant $J$ as a velocity space coordinate and they have been implemented in the orbit-averaged neoclassical code KNOSOS. The equations are used to study the $1/\nu$ regime and the lower collisionality regimes. For generic large aspect ratio stellarators with mirror ratios close to unity, as the collision frequency decreases, the $1/\nu$ regime transitions directly into the $\nu$ regime, without passing through a $\sqrt{\nu}$ regime. An explicit formula for the neoclassical fluxes in the $\nu$ regime is obtained. The formula includes the effect of particles that transition between different types of wells. While these transitions produce stochastic scattering independent of the value of the collision frequency in velocity space, the diffusion in real space remains proportional to the collision frequency. The $\sqrt{\nu}$ regime is only recovered in large aspect ratio stellarators close to omnigeneity: large aspect ratio stellarators with large mirror ratios and optimized large aspect ratio stellarators with mirror ratios close to unity. Neoclassical transport in large aspect ratio stellarators with large mirror ratios can be calculated with the orbit-averaged equations derived by \cite{calvo17}. In these stellarators, the $\sqrt{\nu}$ regime exists in the collisionality interval $\epsilon |\ln \epsilon| \ll \nu_{ii} R a/\rho_i v_{ti} \ll 1/\epsilon$. In optimized large aspect ratio stellarators with mirror ratios close to unity, the $\sqrt{\nu}$ regime occurs in an interval of collisionality that depends on the deviation from omnigeneity $\delta$: $\delta^2 |\ln \delta | \ll \nu_{ii} R a/\rho_i v_{ti} \ll 1$.


title: Deep Learning for Epidemiologists: An Introduction to Neural Networks

abstract: Deep learning methods are increasingly being applied to problems in medicine and healthcare. However, few epidemiologists have received formal training in these methods. To bridge this gap, this article introduces to the fundamentals of deep learning from an epidemiological perspective. Specifically, this article reviews core concepts in machine learning (overfitting, regularization, hyperparameters), explains several fundamental deep learning architectures (convolutional neural networks, recurrent neural networks), and summarizes training, evaluation, and deployment of models. We aim to enable the reader to engage with and critically evaluate medical applications of deep learning, facilitating a dialogue between computer scientists and epidemiologists that will improve the safety and efficacy of applications of this technology.


title: Superconductivity in the Uniform Electron Gas: Irrelevance of Kohn-Luttinger Mechanism

abstract: We study the Cooper instability in jellium model in the controlled regime of small to intermediate values of the Coulomb parameter $r_s \leq 2$. We confirm that superconductivity naturally emerges from purely repulsive interactions described by the Kukkonen-Overhauser vertex function. By employing the implicit renormalization approach we reveal that even in the small-$r_s$ limit, the dominant mechanism behind Cooper instability is based on dynamic screening of the Coulomb interaction--accurately captured by the random phase approximation, whereas the Kohn-Luttinger contribution is negligibly small and, thus, not relevant.


title: Transverse Oscillating Bubble Enhanced Laser-driven Betatron X-ray Radiation Generation

abstract: Ultrafast high-brightness X-ray pulses have proven invaluable for a broad range of research. Such pulses are typically generated via synchrotron emission from relativistic electron bunches using large-scale facilities. Recently, significantly more compact X-ray sources based on laser-wakefield accelerated (LWFA) electron beams have been demonstrated. In particular, laser-driven sources, where the radiation is generated by transverse oscillations of electrons within the plasma accelerator structure (so-called betatron oscillations) can generate highly-brilliant ultrashort X-ray pulses using a comparably simple setup. Here, we experimentally demonstrate a method to markedly enhance and control the parameters of LWFA-driven betatron X-ray emission. With our novel Transverse Oscillating Bubble Enhanced Betatron Radiation (TOBER) scheme, we show a significant increase in the number of generated photons by specifically manipulating the amplitude of the betatron oscillations. We realize this through an orchestrated evolution of the temporal laser pulse shape and the accelerating plasma structure. This leads to controlled off-axis injection of electrons that perform large-amplitude collective transverse betatron oscillations, resulting in increased radiation emission. Our concept holds the promise for a method to optimize the X-ray parameters for specific applications, such as time-resolved investigations with spatial and temporal atomic resolution or advanced high-resolution imaging modalities, and the generation of X-ray beams with even higher peak and average brightness.


title: How to Improve Deep Learning for Software Analytics (a case study with code smell detection)

abstract: To reduce technical debt and make code more maintainable, it is important to be able to warn programmers about code smells. State-of-the-art code small detectors use deep learners, without much exploration of alternatives within that technology. One promising alternative for software analytics and deep learning is GHOST (from TSE'21) that relies on a combination of hyper-parameter optimization of feedforward neural networks and a novel oversampling technique to deal with class imbalance. The prior study from TSE'21 proposing this novel "fuzzy sampling" was somewhat limited in that the method was tested on defect prediction, but nothing else. Like defect prediction, code smell detection datasets have a class imbalance (which motivated "fuzzy sampling"). Hence, in this work we test if fuzzy sampling is useful for code smell detection. The results of this paper show that we can achieve better than state-of-the-art results on code smell detection with fuzzy oversampling. For example, for "feature envy", we were able to achieve 99+\% AUC across all our datasets, and on 8/10 datasets for "misplaced class". While our specific results refer to code smell detection, they do suggest other lessons for other kinds of analytics. For example: (a) try better preprocessing before trying complex learners (b) include simpler learners as a baseline in software analytics (c) try "fuzzy sampling" as one such baseline.


title: PanoDepth: A Two-Stage Approach for Monocular Omnidirectional Depth Estimation

abstract: Omnidirectional 3D information is essential for a wide range of applications such as Virtual Reality, Autonomous Driving, Robotics, etc. In this paper, we propose a novel, model-agnostic, two-stage pipeline for omnidirectional monocular depth estimation. Our proposed framework PanoDepth takes one 360 image as input, produces one or more synthesized views in the first stage, and feeds the original image and the synthesized images into the subsequent stereo matching stage. In the second stage, we propose a differentiable Spherical Warping Layer to handle omnidirectional stereo geometry efficiently and effectively. By utilizing the explicit stereo-based geometric constraints in the stereo matching stage, PanoDepth can generate dense high-quality depth. We conducted extensive experiments and ablation studies to evaluate PanoDepth with both the full pipeline as well as the individual modules in each stage. Our results show that PanoDepth outperforms the state-of-the-art approaches by a large margin for 360 monocular depth estimation.


title: Polarization angle requirements for CMB B-mode experiments. Application to the LiteBIRD satellite

abstract: A methodology to provide the polarization angle requirements for different sets of detectors, at a given frequency of a CMB polarization experiment, is presented. The uncertainties in the polarization angle of each detector set are related to a given bias on the tensor-to-scalar ratio $r$ parameter. The approach is grounded in using a linear combination of the detector sets to obtain the CMB polarization signal. In addition, assuming that the uncertainties on the polarization angle are in the small angle limit (lower than a few degrees), it is possible to derive analytic expressions to establish the requirements. The methodology also accounts for possible correlations among detectors, that may originate from the optics, wafers, etc. The approach is applied to the LiteBIRD space mission. We show that, for the most restrictive case (i.e., full correlation of the polarization angle systematics among detector sets), the requirements on the polarization angle uncertainties are of around 1 arcmin at the most sensitive frequency bands (i.e., $\approx 150$ GHz) and of few tens of arcmin at the lowest (i.e., $\approx 40$ GHz) and highest (i.e., $\approx 400$ GHz) observational bands. Conversely, for the least restrictive case (i.e., no correlation of the polarization angle systematics among detector sets), the requirements are $\approx 5$ times less restrictive than for the previous scenario. At the global and the telescope levels, polarization angle knowledge of a few arcmins is sufficient for correlated global systematic errors and can be relaxed by a factor of two for fully uncorrelated errors in detector polarization angle. The reported uncertainty levels are needed in order to have the bias on $r$ due to systematics below the limit established by the LiteBIRD collaboration.


title: On the extension of quasiplurisubharmonic functions

abstract: Let $(V,\omega)$ be a compact K\"ahler manifold such that $V$ admits a cover by Zariski-open Stein sets with the property that $\omega$ has a strictly plurisubharmonic exhaustive potential on each element of the cover. If $X\subset V$ is an analytic subvariety, we prove that any $\omega|_X$-plurisubharmonic function on $X$ extends to a $\omega$-plurisubharmonic function on $V$. This result generalizes a previous result of ours on the extension of singular metrics of ample line bundles. It allows one to show that any transcendental K\"ahler class in the real Neron-Severi space $NS_{\mathbb R}(V)$ has this extension property.


title: Singlet Fission Photovoltaics: Progress and Promising Pathways

abstract: Singlet fission is a form of multiple exciton generation which occurs in organic chromophores when a high energy singlet exciton separates into two lower energy triplet excitons, each with approximately half the singlet energy. Since this process is spin-allowed it can proceed on an ultrafast timescale of less than several picoseconds, outcompeting most other loss mechanisms and reaching quantitative yields approaching 200%. Due to this high quantum efficiency, the singlet fission process shows promise as a means of reducing thermalisation losses in photovoltaic cells. This would potentially allow for efficiency improvements beyond the thermodynamic limit in a single junction cell. Efforts to incorporate this process into solar photovoltaic cells have spanned a wide range of device structures over the past decade. In this review we compare and categorise these attempts in order to assess the state of the field and identify the most promising avenues of future research and development.


title: Adaptive Sampling Strategies to Construct Equitable Training Datasets

abstract: In domains ranging from computer vision to natural language processing, machine learning models have been shown to exhibit stark disparities, often performing worse for members of traditionally underserved groups. One factor contributing to these performance gaps is a lack of representation in the data the models are trained on. It is often unclear, however, how to operationalize representativeness in specific applications. Here we formalize the problem of creating equitable training datasets, and propose a statistical framework for addressing this problem. We consider a setting where a model builder must decide how to allocate a fixed data collection budget to gather training data from different subgroups. We then frame dataset creation as a constrained optimization problem, in which one maximizes a function of group-specific performance metrics based on (estimated) group-specific learning rates and costs per sample. This flexible approach incorporates preferences of model-builders and other stakeholders, as well as the statistical properties of the learning task. When data collection decisions are made sequentially, we show that under certain conditions this optimization problem can be efficiently solved even without prior knowledge of the learning rates. To illustrate our approach, we conduct a simulation study of polygenic risk scores on synthetic genomic data -- an application domain that often suffers from non-representative data collection. We find that our adaptive sampling strategy outperforms several common data collection heuristics, including equal and proportional sampling, demonstrating the value of strategic dataset design for building equitable models.


title: Goodman surgery and projectively Anosov flows

abstract: We introduce a generalization of Goodman surgery to the category of projectively Anosov flows. This construction is performed along a knot that is simultaneously Legendrian and transverse for a supporting bi-contact structure. If the flow is Anosov there is a particular class of supporting bi-contact structures that induce Lorentzian metrics satisfying Barbot's criterion of hyperbolicity. Foulon and Hasselblatt construct new contact Anosov flows by surgery from a geodesic flow. We generalize their result showing that in any contact Anosov flow there is a family of Legendrian knots that can be used to produce new contact Anosov flows by surgery. Outside of the realm of Anosov flows we generate new examples of projectively Anosov flows on hyperbolic 3-manifolds. These flows contain an invariant submanifold of genus g>0. We also give some application to contact geometry: we interpret the bi-contact surgery in terms of classic contact-Legendrian surgery and admissible-inadmissible transverse surgery and we deduce some (hyper)tightness result for contact and transverse surgeries.


title: Black hole sensitivities in Einstein-scalar-Gauss-Bonnet gravity

abstract: The post-Newtonian dynamics of black hole binaries in Einstein-scalar-Gauss-Bonnet theories of gravity depends on the so-called "sensitivities", quantities which characterize a black hole's adiabatic response to the time-dependent scalar field environment sourced by its companion. In this work, we calculate numerically the sensitivities of nonrotating black holes, including spontaneously scalarized ones, in three classes of Einstein-scalar-Gauss-Bonnet gravity: the shift-symmetric, dilatonic and Gaussian theories. When possible, we compare our results against perturbative analytical results, finding excellent agreement. Unlike their general relativistic counterparts, black holes in Einstein-scalar-Gauss-Bonnet gravity only exist in a restricted parameter space controlled by the theory's coupling constant. A preliminary study of the role played by the sensitivities in black hole binaries suggests that, in principle, black holes can be driven outside of their domain of existence during the inspiral, for binary parameters which we determine.


title: Optical and JWST Mid-IR Emission Line Diagnostics for Simultaneous IMBH and Stellar Excitation in z~0 Dwarf Galaxies

abstract: Current observational facilities have yet to conclusively detect $10^3 - 10^4 M_{\odot}$ intermediate mass black holes (IMBHs) that fill in the evolutionary gap between early universe seed black holes and $z \sim 0$ supermassive black holes. Dwarf galaxies present an opportunity to reveal active IMBHs amidst persistent star formation. We introduce photoionization simulations tailored to address key physical uncertainties: coincident vs. non-coincident mixing of IMBH and starlight excitation, open vs. closed surrounding gas cloud geometries, and different AGN SED shapes. We examine possible AGN emission line diagnostics in the optical and mid-IR, and find that the diagnostics are often degenerate with respect to the investigated physical uncertainties. In spite of these setbacks, and in contrast to recent work, we are able to show that [O III]/H$\beta$ typically remains bright for dwarf AGN powered by IMBHs down to $10^3 M_{\odot}$. Dwarf AGN are predicted to have inconsistent star-forming and Seyfert/LINER classifications using the most common optical diagnostics. In the mid-IR, [O IV] 25.9$\mu$m and [Ar II] 6.98$\mu$m are less sensitive to physical uncertainties than are optical diagnostics. Based on these emission lines, we provide several mid-IR emission line diagnostic diagrams with demarcations for separating starbursts and AGN with varying levels of activity. The diagrams are valid over a wide range of ionization parameters and metallicities out to $z\sim0.1$, so will prove useful for future JWST observations of local dwarf AGN in the search for IMBHs. We make our photoionization simulation suite freely available.


title: Fast Convex Optimization for Two-Layer ReLU Networks: Equivalent Model Classes and Cone Decompositions

abstract: We develop fast algorithms and robust software for convex optimization of two-layer neural networks with ReLU activation functions. Our work leverages a convex reformulation of the standard weight-decay penalized training problem as a set of group-$\ell_1$-regularized data-local models, where locality is enforced by polyhedral cone constraints. In the special case of zero-regularization, we show that this problem is exactly equivalent to unconstrained optimization of a convex "gated ReLU" network. For problems with non-zero regularization, we show that convex gated ReLU models obtain data-dependent approximation bounds for the ReLU training problem. To optimize the convex reformulations, we develop an accelerated proximal gradient method and a practical augmented Lagrangian solver. We show that these approaches are faster than standard training heuristics for the non-convex problem, such as SGD, and outperform commercial interior-point solvers. Experimentally, we verify our theoretical results, explore the group-$\ell_1$ regularization path, and scale convex optimization for neural networks to image classification on MNIST and CIFAR-10.


title: Training a Bidirectional GAN-based One-Class Classifier for Network Intrusion Detection

abstract: The network intrusion detection task is challenging because of the imbalanced and unlabeled nature of the dataset it operates on. Existing generative adversarial networks (GANs), are primarily used for creating synthetic samples from reals. They also have been proved successful in anomaly detection tasks. In our proposed method, we construct the trained encoder-discriminator as a one-class classifier based on Bidirectional GAN (Bi-GAN) for detecting anomalous traffic from normal traffic other than calculating expensive and complex anomaly scores or thresholds. Our experimental result illustrates that our proposed method is highly effective to be used in network intrusion detection tasks and outperforms other similar generative methods on the NSL-KDD dataset.


title: On Automorphism Groups of Idempotent Evolution Algebras

abstract: We study the automorphism group of an idempotent evolution algebra, show that any finite group can be the automorphism group of an evolution algebra, and describe certain evolution algebras with given automorphism groups. In particular, we classify $n$-dimensional idempotent evolution algebras whose automorphism group is isomorphic to the symmetric group $S_n$, and classify idempotent evolution algebras with maximal diagonal automorphism subgroups.


title: Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantization

abstract: Vector Quantization (VQ) is a method for discretizing latent representations and has become a major part of the deep learning toolkit. It has been theoretically and empirically shown that discretization of representations leads to improved generalization, including in reinforcement learning where discretization can be used to bottleneck multi-agent communication to promote agent specialization and robustness. The discretization tightness of most VQ-based methods is defined by the number of discrete codes in the representation vector and the codebook size, which are fixed as hyperparameters. In this work, we propose learning to dynamically select discretization tightness conditioned on inputs, based on the hypothesis that data naturally contains variations in complexity that call for different levels of representational coarseness. We show that dynamically varying tightness in communication bottlenecks can improve model performance on visual reasoning and reinforcement learning tasks.


title: Recommendations for Visualization Recommendations: Exploring Preferences and Priorities in Public Health

abstract: The promise of visualization recommendation systems is that analysts will be automatically provided with relevant and high-quality visualizations that will reduce the work of manual exploration or chart creation. However, little research to date has focused on what analysts value in the design of visualization recommendations. We interviewed 18 analysts in the public health sector and explored how they made sense of a popular in-domain dataset. in service of generating visualizations to recommend to others. We also explored how they interacted with a corpus of both automatically- and manually-generated visualization recommendations, with the goal of uncovering how the design values of these analysts are reflected in current visualization recommendation systems. We find that analysts champion simple charts with clear takeaways that are nonetheless connected with existing semantic information or domain hypotheses. We conclude by recommending that visualization recommendation designers explore ways of integrating context and expectation into their systems.


title: Exploring Transformer Backbones for Heterogeneous Treatment Effect Estimation

abstract: Previous works on Treatment Effect Estimation (TEE) are not in widespread use because they are predominantly theoretical, where strong parametric assumptions are made but untractable for practical application. Recent work uses multilayer perceptron (MLP) for modeling casual relationships, however, MLPs lag far behind recent advances in ML methodology, which limits their applicability and generalizability. To extend beyond the single domain formulation and towards more realistic learning scenarios, we explore model design spaces beyond MLPs, i.e., transformer backbones, which provide flexibility where attention layers govern interactions among treatments and covariates to exploit structural similarities of potential outcomes for confounding control. Through careful model design, Transformers as Treatment Effect Estimators (TransTEE) is proposed. We show empirically that TransTEE can: (1) serve as a general purpose treatment effect estimator that significantly outperforms competitive baselines in a variety of challenging TEE problems (e.g., discrete, continuous, structured, or dosage-associated treatments) and is applicable to both when covariates are tabular and when they consist of structural data (e.g., texts, graphs); (2) yield multiple advantages: compatibility with propensity score modeling, parameter efficiency, robustness to continuous treatment value distribution shifts, explainable in covariate adjustment, and real-world utility in auditing pre-trained language models


title: Generalizability of Machine Learning Models: Quantitative Evaluation of Three Methodological Pitfalls

abstract: Purpose: Despite the potential of machine learning models, the lack of generalizability has hindered their widespread adoption in clinical practice. We investigate three methodological pitfalls: (1) violation of independence assumption, (2) model evaluation with an inappropriate performance indicator or baseline for comparison, and (3) batch effect. Materials and Methods: Using several retrospective datasets, we implement machine learning models with and without the pitfalls to quantitatively illustrate these pitfalls' effect on model generalizability. Results: Violation of independence assumption, more specifically, applying oversampling, feature selection, and data augmentation before splitting data into train, validation, and test sets, respectively, led to misleading and superficial gains in F1 scores of 71.2% in predicting local recurrence and 5.0% in predicting 3-year overall survival in head and neck cancer as well as 46.0% in distinguishing histopathological patterns in lung cancer. Further, randomly distributing data points for a subject across training, validation, and test sets led to a 21.8% superficial increase in F1 score. Also, we showed the importance of the choice of performance measures and baseline for comparison. In the presence of batch effect, a model built for pneumonia detection led to F1 score of 98.7%. However, when the same model was applied to a new dataset of normal patients, it only correctly classified 3.86% of the samples. Conclusions: These methodological pitfalls cannot be captured using internal model evaluation, and the inaccurate predictions made by such models may lead to wrong conclusions and interpretations. Therefore, understanding and avoiding these pitfalls is necessary for developing generalizable models.


title: Regression Transformer: Concurrent sequence regression and generation for molecular language modeling

abstract: Despite significant progress of generative models in the natural sciences, their controllability remains challenging. One fundamentally missing aspect of molecular or protein generative models is an inductive bias that can reflect continuous properties of interest. To that end, we propose the Regression Transformer (RT), a novel method that abstracts regression as a conditional sequence modeling problem. This introduces a new paradigm of multitask language models which seamlessly bridge sequence regression and conditional sequence generation. We thoroughly demonstrate that, despite using a nominal-scale training objective, the RT matches or surpasses the performance of conventional regression models in property prediction tasks of small molecules, proteins and chemical reactions. Critically, priming the same model with continuous properties yields a highly competitive conditional generative model that outperforms specialized approaches in a substructure-constrained, property-driven molecule generation benchmark. Our dichotomous approach is facilitated by a novel, alternating training scheme that enables the model to decorate seed sequences by desired properties, e.g., to optimize reaction yield. In sum, the RT is the first report of a multitask model that concurrently excels at predictive and generative tasks in biochemistry. This finds particular application in property-driven, local exploration of the chemical or protein space and could pave the road toward foundation models in material design. The code to reproduce all experiments of the paper is available at: https://github.com/IBM/regression-transformer


title: Understanding Cross-Domain Few-Shot Learning Based on Domain Similarity and Few-Shot Difficulty

abstract: Cross-domain few-shot learning (CD-FSL) has drawn increasing attention for handling large differences between the source and target domains--an important concern in real-world scenarios. To overcome these large differences, recent works have considered exploiting small-scale unlabeled data from the target domain during the pre-training stage. This data enables self-supervised pre-training on the target domain, in addition to supervised pre-training on the source domain. In this paper, we empirically investigate which pre-training is preferred based on domain similarity and few-shot difficulty of the target domain. We discover that the performance gain of self-supervised pre-training over supervised pre-training becomes large when the target domain is dissimilar to the source domain, or the target domain itself has low few-shot difficulty. We further design two pre-training schemes, mixed-supervised and two-stage learning, that improve performance. In this light, we present six findings for CD-FSL, which are supported by extensive experiments and analyses on three source and eight target benchmark datasets with varying levels of domain similarity and few-shot difficulty. Our code is available at https://github.com/sungnyun/understanding-cdfsl.


title: An Artificial Intelligence Dataset for Solar Energy Locations in India

abstract: Rapid development of renewable energy sources, particularly solar photovoltaics (PV), is critical to mitigate climate change. As a result, India has set ambitious goals to install 500 gigawatts of solar energy capacity by 2030. Given the large footprint projected to meet renewables energy targets, the potential for land use conflicts over environmental values is high. To expedite development of solar energy, land use planners will need access to up-to-date and accurate geo-spatial information of PV infrastructure. In this work, we developed a spatially explicit machine learning model to map utility-scale solar projects across India using freely available satellite imagery with a mean accuracy of 92%. Our model predictions were validated by human experts to obtain a dataset of 1363 solar PV farms. Using this dataset, we measure the solar footprint across India and quantified the degree of landcover modification associated with the development of PV infrastructure. Our analysis indicates that over 74% of solar development In India was built on landcover types that have natural ecosystem preservation, or agricultural value.