title: Can You Learn Semantics Through Next-Word Prediction? The Case of Entailment

abstract: Do LMs infer the semantics of text from co-occurrence patterns in their training data? Merrill et al. (2022) argue that, in theory, probabilities predicted by an optimal LM encode semantic information about entailment relations, but it is unclear whether neural LMs trained on corpora learn entailment in this way because of strong idealizing assumptions made by Merrill et al. In this work, we investigate whether their theory can be used to decode entailment judgments from neural LMs. We find that a test similar to theirs can decode entailment relations between natural sentences, well above random chance, though not perfectly, across many datasets and LMs. This suggests LMs implicitly model aspects of semantics to predict semantic effects on sentence co-occurrence patterns. However, we find the test that predicts entailment in practice works in the opposite direction to the theoretical test. We thus revisit the assumptions underlying the original test, finding its derivation did not adequately account for redundancy in human-written text. We argue that correctly accounting for redundancy related to explanations might derive the observed flipped test and, more generally, improve linguistic theories of human speakers.


title: Advancing Audio Fingerprinting Accuracy Addressing Background Noise and Distortion Challenges

abstract: Audio fingerprinting, exemplified by pioneers like Shazam, has transformed digital audio recognition. However, existing systems struggle with accuracy in challenging conditions, limiting broad applicability. This research proposes an AI and ML integrated audio fingerprinting algorithm to enhance accuracy. Built on the Dejavu Project's foundations, the study emphasizes real-world scenario simulations with diverse background noises and distortions. Signal processing, central to Dejavu's model, includes the Fast Fourier Transform, spectrograms, and peak extraction. The "constellation" concept and fingerprint hashing enable unique song identification. Performance evaluation attests to 100% accuracy within a 5-second audio input, with a system showcasing predictable matching speed for efficiency. Storage analysis highlights the critical space-speed trade-off for practical implementation. This research advances audio fingerprinting's adaptability, addressing challenges in varied environments and applications.


title: Improving threshold for fault-tolerant color code quantum computing by flagged weight optimization

abstract: Color codes are promising quantum error correction (QEC) codes because they have an advantage over surface codes in that all Clifford gates can be implemented transversally. However, thresholds of color codes under circuit-level noise are relatively low mainly because measurements of their high-weight stabilizer generators cause an increase in a circuit depth, and thus, substantial errors are introduced. This makes color codes not the best candidate. Here, we propose a method to suppress the impact of such errors by optimizing weights of decoders using flag qubits and reducing the circuit depth using cat states. We set the weights based on conditional error probabilities conditioned on the measurement outcomes of flag qubits. In numerical simulations, we improve the threshold of the (4.8.8) color code under the circuit-level noise from 0.14% to around 0.27%, which is calculated by using an integer programming decoder. Furthermore, in the (6.6.6) color code, we achieved a circuit-level threshold of around 0.36%, which is almost the same value as the highest value in the previous studies employing the same noise model. In both cases, the achieved logical error rates at low physical error rates are almost one order of magnitude lower than a conventional method that uses a single ancilla qubit for each stabilizer measurement. This method can also be applied to other weight-based decoders, making the color codes more promising for the candidate of experimental implementation of QEC. Furthermore, one can utilize this approach to improve a threshold of wider classes of QEC codes, such as high-rate quantum low-density parity check codes.


title: Retention Induced Biases in a Recommendation System with Heterogeneous Users

abstract: I examine a conceptual model of a recommendation system (RS) with user inflow and churn dynamics. When inflow and churn balance out, the user distribution reaches a steady state. Changing the recommendation algorithm alters the steady state and creates a transition period. During this period, the RS behaves differently from its new steady state. In particular, A/B experiment metrics obtained in transition periods are biased indicators of the RS's long term performance. Scholars and practitioners, however, often conduct A/B tests shortly after introducing new algorithms to validate their effectiveness. This A/B experiment paradigm, widely regarded as the gold standard for assessing RS improvements, may consequently yield false conclusions. I also briefly discuss the data bias caused by the user retention dynamics.


title: Evaluating Ground State Energies of Chemical Systems with Low-Depth Quantum Circuits and High Accuracy

abstract: Solving electronic structure problems is considered one of the most promising applications of quantum computing. However, due to limitations imposed by the coherence time of qubits in the Noisy Intermediate Scale Quantum (NISQ) era or the capabilities of early fault-tolerant quantum devices, it is vital to design algorithms with low-depth circuits. In this work, we develop an enhanced Variational Quantum Eigensolver (VQE) ansatz based on the Qubit Coupled Cluster (QCC) approach, which demands optimization over only $n$ parameters rather than the usual $n+2m$ parameters, where $n$ represents the number of Pauli string time evolution gates $e^{-itP}$, and $m$ is the number of qubits involved. We evaluate the ground state energies of $\mathrm{O_3}$, $\mathrm{Li_4}$, and $\mathrm{Cr_2}$, using CAS(2,2), (4,4) and (6,6) respectively in conjunction with our enhanced QCC ansatz, UCCSD (Unitary Coupled Cluster Single Double) ansatz, and canonical CCSD method as the active space solver, and compare with CASCI results. Finally, we assess our enhanced QCC ansatz on two distinct quantum hardware, IBM Kolkata and Quantinuum H1-1.


title: New directions in algebraic statistics: Three challenges from 2023

abstract: In the last quarter of a century, algebraic statistics has established itself as an expanding field which uses multilinear algebra, commutative algebra, computational algebra, geometry, and combinatorics to tackle problems in mathematical statistics. These developments have found applications in a growing number of areas, including biology, neuroscience, economics, and social sciences. Naturally, new connections continue to be made with other areas of mathematics and statistics. This paper outlines three such connections: to statistical models used in educational testing, to a classification problem for a family of nonparametric regression models, and to phase transition phenomena under uniform sampling of contingency tables. We illustrate the motivating problems, each of which is for algebraic statistics a new direction, and demonstrate an enhancement of related methodologies.


title: A robust near-field body area network based on coaxially-shielded textile metamaterial

abstract: A body area network (BAN) involving wearable sensors populated around the human body can continuously monitor physiological signals, finding applications in personal healthcare and athletic evaluation. Existing near-field communication (NFC)-enabled BAN solutions, while facilitating reliable and secure interconnection among battery-free sensors, face challenges such as limited spectral stability against external interference. Here we demonstrate a textile metamaterial featuring a coaxially-shielded internal structure designed to mitigate interference from extraneous loadings. The metamaterial can be patterned onto clothing to form a scalable, customizable network, enabling communication between NFC-enabled devices and developed battery-free textile NFC sensing nodes placed within the network. Proof of concept demonstration shows the metamaterial's robustness against mechanical deformation and exposure to lossy, conductive saline solutions, underscoring its potential applications in wet environments, particularly in athletic activities involving water or significant perspiration, offering insights for the future development of radio frequency components for a robust BAN at the system level.


title: Towards Building Multilingual Language Model for Medicine

abstract: The development of open-source, multilingual medical language models can benefit a wide, linguistically diverse audience from different regions. To promote this domain, we present contributions from the following: First, we construct a multilingual medical corpus, containing approximately 25.5B tokens encompassing 6 main languages, termed as MMedC, enabling auto-regressive domain adaptation for general LLMs; Second, to monitor the development of multilingual medical LLMs, we propose a multilingual medical multi-choice question-answering benchmark with rationale, termed as MMedBench; Third, we have assessed a number of open-source large language models (LLMs) on our benchmark, along with those further auto-regressive trained on MMedC. Our final model, MMed-Llama 3, with only 8B parameters, achieves superior performance compared to all other open-source models on both MMedBench and English benchmarks, even rivaling GPT-4. In conclusion, in this work, we present a large-scale corpus, a benchmark and a series of models to support the development of multilingual medical LLMs.


title: Switchback Patches Evolve into Microstreams via Magnetic Relaxation

abstract: Magnetic switchbacks are distinct magnetic structures characterized by their abrupt reversal in the radial component of the magnetic field within the pristine solar wind. Switchbacks are believed to lose magnetic energy with heliocentric distance. To investigate this switchbacks originating from similar solar source regions are identified during a radial alignment of the Parker Solar Probe (PSP; 25.8 solar radii) and Solar Orbiter (SolO; 152 solar radii). We found that 1) the dynamic and thermal pressures decrease at the switchback boundaries by up to 20% at PSP and relatively unchanged at SolO and magnetic pressure jump across the boundary remains negligible at both distances, and 2) bundles of switchbacks are often observed in switchback patches near the Sun, and in microstreams farther away. Background proton velocity (vp) is 10% greater than the pristine solar wind (vsw) in microstreams, whereas vp ~ vsw in switchback patches. Microstreams contain an average of 30% fewer switchbacks than switchback patches. It is concluded that switchbacks likely relax magnetically and equilibrate their plasma with the surrounding environment with heliocentric distance. Switchback relaxation can, in turn, accelerate the surrounding plasma. Therefore, it is hypothesized that magnetic relaxation of switchbacks may cause switchback patches to evolve into microstreams with heliocentric distance. Statistical analysis of PSP and SolO switchbacks is underway to further test our hypothesis.


title: Cybersecurity as a Service

abstract: With the increasing sophistication and sheer number of cyberattacks, more and more companies come to the conclusion that they have to strengthen their cybersecurity posture. At the same time, well-educated Information technology (IT) security personnel are scarce. Cybersecurity as a service (CSaaS) is one possible solution to tackle this problem by outsourcing security functions to managed security service providers (MSSP). This chapter gives an overview of common CSaaS functions and their providers. Moreover, it provides guidance especially for small- and medium-sized businesses, for asking the appropriate questions when it comes to the selection of a specific MSSP.


title: Study of nanodiamond photocathodes for MPGD-based detectors of single photons

abstract: The proposed new Electron-Ion Collider poses a technical and intellectual challenge for the detector design to accommodate the long-term diverse physics goals envisaged by the program. This requires a 4{\pi} detector system capable of reconstructing the energy and momentum of final state particles with high precision. The Electron-Ion Collider also requires identification of particles of different masses over a wide momentum range. A diverse spectrum of Particle Identification detectors has been proposed. Of the four types of detectors for hadron identification, three are based on Ring Imaging Cherenkov Counter technologies, and one is realized by the Time of Flight method. The quest for a novel photocathode, sensitive in the far vacuum ultra violet wavelength range and more robust than cesium iodide, motivated an R&D programme to explore nano-diamond (ND) based photocathodes, started by a collaboration between INFN and CNR Bari and INFN Trieste. Systematic measurements of the photo emission in different Ar-CH4 and Ar-CO2 gas mixtures with various types of ND powders and Hydrogenated ND (H-ND) powders are reported. A first study of the response of THGEMs coated with different photocathode materials is presented. The progress of this R&D programme and the results obtained so far by these exploratory studies are described.


title: Neutrino phenomenology and keV dark matter in 2HDM with $A_4$ symmetry

abstract: We propose a minimal extended seesaw scheme based on the discrete symmetry $A_4\times Z_4\times Z_2\times Z_8$ which can successfully address neutrino phenomenology and keV sterile neutrino dark matter. The lepton mass hierarchy is naturally achieved. Active neutrino mixing angles can reached the best-fit points with the predictive Dirac CP violation phase. The active-sterile mixing matrix elements are small enough to access the observed cosmological dark matter abundance constraint with keV sterile neutrino dark matter. The effective neutrino masses are predicted to be in the ranges of the recent experimental limits.


title: On the decomposition group of a nonsingular plane cubic by a log Calabi-Yau geometrical perspective

abstract: This paper aims to study the decomposition group of a nonsingular plane cubic under the light of the log Calabi-Yau geometry. Using this approach we prove that an appropriate algorithm of the Sarkisov Program in dimension 2 applied to an element of this group is automatically volume preserving. From this, we deduce some properties of the (volume preserving) Sarkisov factorization of its elements. We also negatively answer a question posed by Blanc, Pan and Vust asking whether the canonical complex of a nonsingular plane cubic is split. Within a similar context em dimension 3, we exhibit in detail an interesting counterexample for a possible generalization of a theorem by Pan in which there exists a Sarkisov factorization obtained algorithmically that is not volume preserving.


title: On modular representations of inner forms of $\mathrm{GL}_n$ over a local non-archimedean field

abstract: Let $\mathrm{F}$ be a local non-archimedean field of residue characteristic $p$ and $\overline{\mathbb{F}}_\ell$ an algebraic closure of a finite field of characteristic $\ell \neq p$. We extend the results of Lapid and M\'inguez concerning $\square$-irreducible representations of inner forms of $\mathrm{GL}_n(\mathrm{F})$ to representations over $\overline{\mathbb{F}}_\ell$. As applications, we compute the Godement-Jacquet $L$-factor for any smooth irreducible representation over $\overline{\mathbb{F}}_\ell$ and show that the local factors of a representation agree with the ones of its $\mathrm{C}$-parameter defined by Kurinczuk and Matringe. Moreover, we reprove that the classification of irreducible representations via multisegments due to Vign\'eras and M\'inguez-S\'echerre is indeed exhaustive without using the classification of Ariki and Mathas of simple modules of Hecke algebras. Finally, we characterize the irreducible constituents of certain parabolically induced representations, as was already done by Zelevinsky over $\mathbb{C}$.


title: Birational geometry of Calabi-Yau pairs $(\mathbb{P}^3, D)$ of coregularity 2

abstract: This paper aims to study the birational geometry of log Calabi-Yau pairs$(\mathbb{P}^3, D)$ of coregularity 2, where in this case $D$ is an irreducible normal quartic surface with canonical singularities. We completely classify which toric weighted blowups of a point will initiate a volume preserving Sarkisov link starting with this pair. Depending on the type of singularity, our results point out that some of these weights do not work generically for a general member of the corresponding coarse moduli space of quartics.


title: Multi-indice B-series

abstract: We propose a novel way to describe numerical methods for ordinary differential equations via the notion of multi-indice. The main idea is to replace rooted trees in Butcher's B-series by multi-indices. The latter were introduced recently in the context of describing solutions of singular stochastic partial differential equations. The combinatorial shift away from rooted trees allows for a compressed description of numerical schemes. Moreover, these multi-indices B-series characterise uniquely the Taylor development of local and affine equivariant maps.


title: Two-dimensional Lorentz-violating Casimir effect

abstract: In this study, we consider the four-dimensional Maxwell electrodynamics extended with CPT-even Myers-Pospelov Lorentz-violating dimension-six operators to investigate the associated two-dimensional properties in the context of quantum vacuum fluctuation effects, namely, the Casimir effect. Upon projecting out the 4D theory down to a 2D theory we obtain analogs of these operators leading to a modified dispersion relation in a Lorentz invariance violation (LIV) scalar model equivalent to the electromagnetic theory. By making use of the modified dispersion relation, we derive exact analytic expressions for the Casimir energy and force induced by imposing Dirichlet boundary conditions on the scalar field. In the regime where the LIV parameter becomes very small, we recover known results for the Casimir energy and force plus correction terms due to the LIV.


title: Linear-Time Graph Neural Networks for Scalable Recommendations

abstract: In an era of information explosion, recommender systems are vital tools to deliver personalized recommendations for users. The key of recommender systems is to forecast users' future behaviors based on previous user-item interactions. Due to their strong expressive power of capturing high-order connectivities in user-item interaction data, recent years have witnessed a rising interest in leveraging Graph Neural Networks (GNNs) to boost the prediction performance of recommender systems. Nonetheless, classic Matrix Factorization (MF) and Deep Neural Network (DNN) approaches still play an important role in real-world large-scale recommender systems due to their scalability advantages. Despite the existence of GNN-acceleration solutions, it remains an open question whether GNN-based recommender systems can scale as efficiently as classic MF and DNN methods. In this paper, we propose a Linear-Time Graph Neural Network (LTGNN) to scale up GNN-based recommender systems to achieve comparable scalability as classic MF approaches while maintaining GNNs' powerful expressiveness for superior prediction accuracy. Extensive experiments and ablation studies are presented to validate the effectiveness and scalability of the proposed algorithm. Our implementation based on PyTorch is available.


title: Quantum spin Hall effect protected by spin U(1) quasi-symmetry

abstract: Quantum spin Hall (QSH) effect, where electrons with opposite spin channels are deflected to opposite sides of a two-dimensional system with a quantized conductance, was believed to be characterized by a nontrivial topological index $Z_{2}$. However, spin mixing effects in realistic materials often lead to deviation of the spin Hall conductance from exact quantization. In this Letter, we present a universal symmetry indicator for diagnosing QSH effect in realistic materials, termed spin $U$(1) quasi-symmetry. Such a symmetry eliminates the first-order spin-mixing perturbation and thus protects the near-quantization of SHC, applicable to time-reversal-preserved cases with either $Z_{2}=1$ or $Z_{2}=0$, as well as time-reversal-broken scenarios. We propose that spin $U$(1) quasi-symmetry is hidden in the subspace spanned by the doublets with unquenched orbital momentum and emerges when SOC is present, which can be realized in 19 crystallographic point groups. Particularly, we identify a previous overlooked even spin Chern phase with a trivial $Z_{2}$ index as an ideal platform for achieving a near-double-quantized SHC, as exemplified by twisted bilayer transition metal dichalcogenides and monolayer RuBr$_{3}$. Our work offers a new perspective for understanding QSH effect and significantly expands the material pool for the screening of exemplary material candidates.


title: A sublinear-time randomized algorithm for column and row subset selection based on strong rank-revealing QR factorizations

abstract: In this work, we analyze a sublinear-time algorithm for selecting a few rows and columns of a matrix for low-rank approximation purposes. The algorithm is based on an initial uniformly random selection of rows and columns, followed by a refinement of this choice using a strong rank-revealing QR factorization. We prove bounds on the error of the corresponding low-rank approximation (more precisely, the CUR approximation error) when the matrix is a perturbation of a low-rank matrix that can be factorized into the product of matrices with suitable incoherence and/or sparsity assumptions.


title: Non-Markovian maximal couplings and a vertical reflection principle on a class of sub-Riemannian manifolds

abstract: We develop non-Markovian, non-co-adapted couplings for sub-Riemannian Brownian motions in various sub-Riemannian manifolds, namely the three-dimensional Heisenberg group, higher-dimensional non-isotropic Heisenberg groups, SL(2,R) and its universal cover, and SU(2,C). Our primary focus is on the situation when the processes start from two points on the same vertical fiber, since in general co-adapted couplings cannot give the sharp rate for the coupling time in this case. Then for general points, we use this vertical coupling as the second stage of a two-stage coupling. Non-Markovian couplings in this context were first introduced by Banerjee-Gordina-Mariano, for the three-dimensional Heisenberg group, and were more recently extended by B\'en\'efice to SL(2,R) and SU(2,C), using a detailed consideration of the Brownian bridge. In contrast, our couplings are based on global isometries of the space, giving couplings that are maximal, as well as making the construction relatively simple and uniform across different manifolds. Moreover, this construction gives a coupling time that equals a hitting time for the vertical component of one of the Brownian motions, and this vertical component satisfies a reflection principle, which is useful in explicitly bounding the tail probability of the coupling time. We estimate the coupling time in these various situations and give applications to inequalities for the heat semigroup.


title: Relative entropy and modulated free energy without confinement via self-similar transformation

abstract: This note extends the modulated entropy and free energy methods for proving mean-field limits/propagation of chaos to the whole space without any confining potential, in contrast to previous work limited to the torus or requiring confinement in the whole space, for all log/Riesz flows. Our novel idea is a scale transformation, sometimes called self-similar coordinates in the PDE literature, which converts the problem to one with a quadratic confining potential, up to a time-dependent renormalization of the interaction potential. In these self-similar coordinates, one can then establish a Gr\"onwall relation for the relative entropy or modulated free energy, conditional on bounds for the Hessian of the mean-field log density. This generalizes recent work of Feng-Wang arXiv:2310.05156, which extended the Jabin-Wang relative entropy method to the whole space for the viscous vortex model. Moreover, in contrast to previous work, our approach allows to obtain uniform-in-time propagation of chaos and even polynomial-in-time generation of chaos in the whole space without confinement, provided one has suitable decay estimates for the mean-field log density. The desired regularity bounds and decay estimates are the subject of a companion paper.


title: Web bases in degree two from hourglass plabic graphs

abstract: Webs give a diagrammatic calculus for spaces of $U_q(\mathfrak{sl}_r)$-tensor invariants, but intrinsic characterizations of web bases are only known in certain cases. Recently, we introduced hourglass plabic graphs to give the first such $U_q(\mathfrak{sl}_4)$-web bases. Separately, Fraser introduced a web basis for Pl\"{u}cker degree two representations of arbitrary $U_q(\mathfrak{sl}_r)$. Here, we show that Fraser's basis agrees with that predicted by the hourglass plabic graph framework and give an intrinsic characterization of the resulting webs. A further compelling feature with many applications is that our bases exhibit rotation-invariance. Together with the results of our earlier paper, this implies that hourglass plabic graphs give a uniform description of all known rotation-invariant $U_q(\mathfrak{sl}_r)$-web bases. Moreover, this provides a single combinatorial model simultaneously generalizing the Tamari lattice, the alternating sign matrix lattice, and the lattice of plane partitions. As a part of our argument, we develop properties of square faces in arbitrary hourglass plabic graphs, a key step in our program towards general $U_q(\mathfrak{sl}_r)$-web bases.


title: The Importance of Architecture Choice in Deep Learning for Climate Applications

abstract: Machine Learning has become a pervasive tool in climate science applications. However, current models fail to address nonstationarity induced by anthropogenic alterations in greenhouse emissions and do not routinely quantify the uncertainty of proposed projections. In this paper, we model the Atlantic Meridional Overturning Circulation (AMOC) which is of major importance to climate in Europe and the US East Coast by transporting warm water to these regions, and has the potential for abrupt collapse. We can generate arbitrarily extreme climate scenarios through arbitrary time scales which we then predict using neural networks. Our analysis shows that the AMOC is predictable using neural networks under a diverse set of climate scenarios. Further experiments reveal that MLPs and Deep Ensembles can learn the physics of the AMOC instead of imitating its progression through autocorrelation. With quantified uncertainty, an intriguing pattern of "spikes" before critical points of collapse in the AMOC casts doubt on previous analyses that predicted an AMOC collapse within this century. Our results show that Bayesian Neural Networks perform poorly compared to more dense architectures and care should be taken when applying neural networks to nonstationary scenarios such as climate projections. Further, our results highlight that big NN models might have difficulty in modeling global Earth System dynamics accurately and be successfully applied in nonstationary climate scenarios due to the physics being challenging for neural networks to capture.


title: Geometry-induced wavefunction collapse

abstract: When a quantum particle moves in a curved space, a geometric potential can arise. In spite of a long history of extensive theoretical studies, to experimentally observe the geometric potential remains to be a challenge. What are the physically observable consequences of such a geometric potential? Solving the Schrodinger equation on a truncated conic surface, we uncover a class of quantum scattering states that bear a strong resemblance with the quasi-resonant states associated with atomic collapse about a Coulomb impurity, a remarkable quantum phenomenon in which an infinite number of quasi-resonant states emerge. A characteristic defining feature of such collapse states is the infinite oscillations of the local density of states (LDOS) about the zero energy point separating the scattering from the bound states. The emergence of such states in the curved (Riemannian) space requires neither a relativistic quantum mechanism nor any Coulomb impurity: they have zero angular momentum and their origin is purely geometrical - henceforth the term geometry-induced wavefunction collapse. We establish the collapsing nature of these states through a detailed comparative analysis of the behavior of the LDOS for both the zero and finite angular-momentum states as well as the corresponding classical picture. Potential experimental schemes to realize the geometry-induced collapse states are articulated. Not only has our study uncovered an intrinsic connection between the geometric potential and atomic collapse, it also provides a method to experimentally observe and characterize geometric potentials arising from different subfields of physics. For example, in nanoscience and nanotechnology, curved geometry has become increasingly common. Our finding suggests that wavefunction collapse should be an important factor of consideration in designing and developing nanodevices.


title: Plug-and-play analytical paradigm for the scattering of plane waves by "layer-cake" periodic systems

abstract: We investigate the scattering of scalar plane waves in two dimensions by a heterogeneous layer that is periodic in the direction parallel to its boundary. On describing the layer as a union of periodic laminae, we develop a solution of the scattering problem by merging the concept of propagator matrices and that of Bloch eigenstates featured by the unit cell of each lamina. The featured Bloch eigenstates are obtained by solving the quadratic eigenvalue problem (QEVP) that seeks a complex-valued wavenumber normal to the layer boundary given (i) the excitation frequency, and (ii) real-valued wavenumber parallel to the boundary -- that is preserved throughout the system. Spectral analysis of the QEVP reveals sufficient conditions for discreteness of the eigenvalue spectrum and the fact that all eigenvalues come in complex-conjugate pairs. By deploying the factorization afforded by the propagator matrix approach, we demonstrate that the contribution of individual eigenvalues (and so eigenmodes) to the solution diminishes exponentially with absolute value of their imaginary part, which then forms a rational basis for truncation of the factorized Bloch-wave solution. The proposed methodology caters for the optimal design of rainbow traps, energy harvesters, and metasurfaces, whose potency to manipulate waves is decided not only by the individual dispersion characteristics of the component laminae, but also by ordering and generally fitting of the latter into a composite layer. Using the factorized Bloch approach, evaluation of trial configurations -- as generated by the permutation and window translation/stretching of the component laminae -- can be accelerated by decades.


title: Specht property for the graded identities of the pair $(M_2(D), sl_2(D))$

abstract: Let $D$ be a Noetherian infinite integral domain, denote by $M_2(D)$ and by $sl_2(D)$ the $2\times 2$ matrix algebra and the Lie algebra of the traceless matrices in $M_2(D)$, respectively. In this paper we study the natural grading by the cyclic group $\mathbb{Z}_2$ of order 2 on $M_2(D)$ and on $sl_2(D)$. We describe a finite basis of the graded polynomial identities for the pair $(M_2(D), sl_2(D))$. Moreover we prove that the ideal of the graded identities for this pair satisfies the Specht property, that is every ideal of graded identities of pairs (associative algebra, Lie algebra), satisfying the graded identities for $(M_2(D), sl_2(D))$, is finitely generated. The polynomial identities for $M_2(D)$ are known if $D$ is any field of characteristic different from 2. The identities for the Lie algebra $sl_2(D)$ are known when $D$ is an infinite field. The identities for the pair we consider were first described by Razmyslov when $D$ is a field of characteristic 0, and afterwards by the second author when $D$ is an infinite field. The graded identities for the pair $(M_2(D), gl_2(D))$ were also described, by Krasilnikov and the second author. In order to obtain these results we use certain graded analogues of the generic matrices, and also techniques developed by G. Higman concerning partially well ordered sets.


title: QCD parameters and SM-high precision from $e^+e^-$ to Hadrons: Updated

abstract: 1. I update my previous comparison of the theoretical value of the muon anomaly with the new measurement and found $\Delta a_\mu\equiv a_\mu^{exp}-a_\mu^{th} = (143\pm 42_{th}\pm 22_{exp})\times 10^{-11}$ which is about 3$\sigma$ discrepancy between the SM predictions and experiment. 2. I improve the estimate of QCD power corrections up to dimension D=12 and provide a new estimate of the ones up to D=20 within the SVZ expansion by combining the ratio of the Laplace sum rules (LSR) with the BNP $\tau$-like decay moments for the I=1 vector current. The results in Table 1 confirm a violation of the factorization of the four-quark condensates and the value of the gluon one $ <\alpha_s G^2>$ from some other sources. Up to D=20, I there is not any factorial nor exponential growth of the size of these power corrections. 3. I use these new values of power corrections to extract $\alpha_s$ from the BNP lowest moment. To order $\alpha_s^4$, I find within the SVZ expansion: $\alpha_s(M_\tau)= 0.3081(50)_{fit}(71)_{\alpha_s^5}$ [resp. $0.3260 (47)_{fit}(62)_{\alpha_s^5}]$ implying $\alpha_s(M_Z)= 0.1170(6)(3)_{evol}$ [resp. $0.1192(6)(3)_{evol}$] for Fixed Order (FO) [resp. Contour Improved (CI)] PT series. They lead to the mean: $ \alpha_s(M_\tau)\vert_{SVZ}=0.3179(58)_{fit}(81)_{syst}$ and $ \alpha_s(M_Z)\vert_{SVZ}= 0.1182(12)(3)_{evol}$ where the systematic error(syst) takes into account the discrepancy between FO and CI. Using the lowest BNP moment, we also obtain from the vector (V) component of $\tau$-decay the mean: $ \alpha_s(M_\tau)\vert_{\tau,V}=0.3219(52)(91)_{syst}$ giving: $\alpha_s(M_Z)\vert_{\tau,V}=0.1187(13)(3)_{evol}$. The average of the two determinations leads to: $ \alpha_s(M_\tau)=0.3198(72)$ and $\alpha_s(M_Z)= 0.1185(9)(3)_{evol}$. 4. Some contributions beyond the SVZ expansion ($1/Q^2$, instantons and duality violation) expected to be small are discussed in Sections 10,11.


title: Stability-Aware Training of Neural Network Interatomic Potentials with Differentiable Boltzmann Estimators

abstract: Neural network interatomic potentials (NNIPs) are an attractive alternative to ab-initio methods for molecular dynamics (MD) simulations. However, they can produce unstable simulations which sample unphysical states, limiting their usefulness for modeling phenomena occurring over longer timescales. To address these challenges, we present Stability-Aware Boltzmann Estimator (StABlE) Training, a multi-modal training procedure which combines conventional supervised training from quantum-mechanical energies and forces with reference system observables, to produce stable and accurate NNIPs. StABlE Training iteratively runs MD simulations to seek out unstable regions, and corrects the instabilities via supervision with a reference observable. The training procedure is enabled by the Boltzmann Estimator, which allows efficient computation of gradients required to train neural networks to system observables, and can detect both global and local instabilities. We demonstrate our methodology across organic molecules, tetrapeptides, and condensed phase systems, along with using three modern NNIP architectures. In all three cases, StABlE-trained models achieve significant improvements in simulation stability and recovery of structural and dynamic observables. In some cases, StABlE-trained models outperform conventional models trained on datasets 50 times larger. As a general framework applicable across NNIP architectures and systems, StABlE Training is a powerful tool for training stable and accurate NNIPs, particularly in the absence of large reference datasets.

categories: cs.LG cond-mat.dis-nn cond-mat.mtrl-sci physics.chem-ph physics.comp-ph


title: On Distributed Computation of the Minimum Triangle Edge Transversal

abstract: The distance of a graph from being triangle-free is a fundamental graph parameter, counting the number of edges that need to be removed from a graph in order for it to become triangle-free. Its corresponding computational problem is the classic minimum triangle edge transversal problem, and its normalized value is the baseline for triangle-freeness testing algorithms. While triangle-freeness testing has been successfully studied in the distributed setting, computing the distance itself in a distributed setting is unknown, to the best of our knowledge, despite being well-studied in the centralized setting. This work addresses the computation of the minimum triangle edge transversal in distributed networks. We show with a simple warm-up construction that this is a global task, requiring $\Omega(D)$ rounds even in the $\mathsf{LOCAL}$ model with unbounded messages, where $D$ is the diameter of the network. However, we show that approximating this value can be done much faster. A $(1+\epsilon)$-approximation can be obtained in $\text{poly}\log{n}$ rounds, where $n$ is the size of the network graph. Moreover, faster approximations can be obtained, at the cost of increasing the approximation factor to roughly 3, by a reduction to the minimum hypergraph vertex cover problem. With a time overhead of the maximum degree $\Delta$, this can also be applied to the $\mathsf{CONGEST}$ model, in which messages are bounded. Our key technical contribution is proving that computing an exact solution is ``as hard as it gets'' in $\mathsf{CONGEST}$, requiring a near-quadratic number of rounds. Because this problem is an edge selection problem, as opposed to previous lower bounds that were for node selection problems, major challenges arise in constructing the lower bound, requiring us to develop novel ingredients.


title: WEAK $G$-IDENTITIES FOR THE PAIR $(M_2( \mathbb{C}),sl_2( \mathbb{C}))$

abstract: In this paper we study algebras acted on by a finite group $G$ and the corresponding $G$-identities. Let $M_2( \mathbb{C})$ be the $2\times 2$ matrix algebra over the field of complex numbers $ \mathbb{C}$ and let $sl_2( \mathbb{C})$ be the Lie algebra of traceless matrices in $M_2( \mathbb{C})$. Assume that $G$ is a finite group acting as a group of automorphisms on $M_2( \mathbb{C})$. These groups were described in the Nineteenth century, they consist of the finite subgroups of $PGL_2( \mathbb{C})$, which are, up to conjugacy, the cyclic groups $ \mathbb{Z}_n$, the dihedral groups $D_n$ (of order $2n$), the alternating groups $ A_4$ and $A_5$, and the symmetric group $S_4$. The $G$-identities for $M_2( \mathbb{C})$ were described by Berele. The finite groups acting on $sl_2( \mathbb{C})$ are the same as those acting on $M_2( \mathbb{C})$. The $G$-identities for the Lie algebra of the traceless $sl_2( \mathbb{C})$ were obtained by Mortari and by the second author. We study the weak $G$-identities of the pair $(M_2( \mathbb{C}), sl_2( \mathbb{C}))$, when $G$ is a finite group. Since every automorphism of the pair is an automorphism for $M_2( \mathbb{C})$, it follows from this that $G$ is one of the groups above. In this paper we obtain bases of the weak $G$-identities for the pair $(M_2( \mathbb{C}), sl_2( \mathbb{C}))$ when $G$ is a finite group acting as a group of automorphisms.


title: A Simple and Yet Fairly Effective Defense for Graph Neural Networks

abstract: Graph Neural Networks (GNNs) have emerged as the dominant approach for machine learning on graph-structured data. However, concerns have arisen regarding the vulnerability of GNNs to small adversarial perturbations. Existing defense methods against such perturbations suffer from high time complexity and can negatively impact the model's performance on clean graphs. To address these challenges, this paper introduces NoisyGNNs, a novel defense method that incorporates noise into the underlying model's architecture. We establish a theoretical connection between noise injection and the enhancement of GNN robustness, highlighting the effectiveness of our approach. We further conduct extensive empirical evaluations on the node classification task to validate our theoretical findings, focusing on two popular GNNs: the GCN and GIN. The results demonstrate that NoisyGNN achieves superior or comparable defense performance to existing methods while minimizing added time complexity. The NoisyGNN approach is model-agnostic, allowing it to be integrated with different GNN architectures. Successful combinations of our NoisyGNN approach with existing defense techniques demonstrate even further improved adversarial defense results. Our code is publicly available at: https://github.com/Sennadir/NoisyGNN.


title: Hamiltonian Descent and Coordinate Hamiltonian Descent

abstract: We propose an optimization algorithm called Hamiltonian Descent, which is a direct counterpart of classical Hamiltonian Monte Carlo in sampling. We find that Hamiltonian Descent for solving strongly convex quadratic problems exhibits a novel update scheme that involves matrix-power-vector products. We also propose Coordinate Hamiltonian Descent and its parallelizable variant, which turns out to encapsulate the classical Gauss-Seidel method, Successive Over-relaxation, Jacobi method, and more, for solving a linear system of equations. The result not only offers a new perspective on these existing algorithms but also leads to a broader class of update schemes that guarantee the convergence.


title: FedADMM-InSa: An Inexact and Self-Adaptive ADMM for Federated Learning

abstract: Federated learning (FL) is a promising framework for learning from distributed data while maintaining privacy. The development of efficient FL algorithms encounters various challenges, including heterogeneous data and systems, limited communication capacities, and constrained local computational resources. Recently developed FedADMM methods show great resilience to both data and system heterogeneity. However, they still suffer from performance deterioration if the hyperparameters are not carefully tuned. To address this issue, we propose an inexact and self-adaptive FedADMM algorithm, termed FedADMM-InSa. First, we design an inexactness criterion for the clients' local updates to eliminate the need for empirically setting the local training accuracy. This inexactness criterion can be assessed by each client independently based on its unique condition, thereby reducing the local computational cost and mitigating the undesirable straggle effect. The convergence of the resulting inexact ADMM is proved under the assumption of strongly convex loss functions. Additionally, we present a self-adaptive scheme that dynamically adjusts each client's penalty parameter, enhancing algorithm robustness by mitigating the need for empirical penalty parameter choices for each client. Extensive numerical experiments on both synthetic and real-world datasets are conducted. As validated by some numerical tests, our proposed algorithm can reduce the clients' local computational load significantly and also accelerate the learning process compared to the vanilla FedADMM.


title: A General Theory of Static Response for Markov Jump Processes

abstract: We consider Markov jump processes on a graph described by a rate matrix that depends on various control parameters. We derive explicit expressions for the static responses of edge currents and steady-state probabilities. We show that they are constrained by the graph topology (i.e. the incidence matrix) by deriving response relations (i.e. linear constraints linking the different responses) and topology-dependent bounds. For unicyclic networks, all scaled current sensitivities are between zero and one and must sum to one. Applying these results to stochastic thermodynamics, we derive explicit expressions for the static response of fundamental currents (which carry the full dissipation) to fundamental thermodynamic forces (which drive the system away from equilibrium).


title: Analysing The Impact of Sequence Composition on Language Model Pre-Training

abstract: Most language model pre-training frameworks concatenate multiple documents into fixed-length sequences and use causal masking to compute the likelihood of each token given its context; this strategy is widely adopted due to its simplicity and efficiency. However, to this day, the influence of the pre-training sequence composition strategy on the generalisation properties of the model remains under-explored. In this work, we find that applying causal masking can lead to the inclusion of distracting information from previous documents during pre-training, which negatively impacts the performance of the models on language modelling and downstream tasks. In intra-document causal masking, the likelihood of each token is only conditioned on the previous tokens in the same document, eliminating potential distracting information from previous documents and significantly improving performance. Furthermore, we find that concatenating related documents can reduce some potential distractions during pre-training, and our proposed efficient retrieval-based sequence construction method, BM25Chunk, can improve in-context learning (+11.6\%), knowledge memorisation (+9.8\%), and context utilisation (+7.2\%) abilities of language models without sacrificing efficiency.


title: Meditating in Live Stream: An Autoethnographic and Interview Study to Investigate Motivations, Interactions and Challenges

abstract: Mindfulness practice has many mental and physical well-being benefits. With the increased popularity of live stream technologies and the impact of COVID-19, many people have turned to live stream tools to participate in online meditation sessions. To better understand the practices, challenges, and opportunities in live-stream meditation, we conducted a three-month autoethnographic study, during which two researchers participated in live-stream meditation sessions as the audience. Then we conducted a follow-up semi-structured interview study with 10 experienced live meditation teachers who use different live-stream tools. We found that live meditation, although having a weaker social presence than in-person meditation, facilitates attendees in establishing a practice routine and connecting with other meditators. Teachers use live streams to deliver the meditation practice to the world which also enhances their practice and brand building. We identified the challenges of using live-stream tools for meditation from the perspectives of both audiences and teachers, and provided design recommendations to better utilize live meditation as a resource for mental wellbeing.


title: Particle detection performance and Geant4 simulation with low-cost CMOS technology

abstract: We analyze the performance of an Omnivision OV5647 CMOS image sensor (5 Mp) for measuring the radiation emitted from Sr90 and Cs137 sources. Our experimental arrangement includes a Raspberry Pi 3 mini-computer for data taking, processed with Python and OpenCV libraries. We specify the camera settings to be sensitive to detecting electrons and photons. We also implement a detailed Geant4 simulation of the CMOS sensor and the radioactive sources. This simulation connects the deposited energy in the pixel matrix by the electrons and photons from the radioactive sources and the ADC counts. Our measurements are expressed through the cluster size, the maximum ADC signal per cluster, and the variation of the clusters with different distances. We find a good agreement between the data and the Geant4 simulation for all these observables. Furthermore, we can reproduce the correlation between the cluster size and the maximum ADC per cluster.


title: Clifford circuits over non-cyclic abelian groups

abstract: We present a discussion of the generalized Clifford group over non-cyclic finite abelian groups. These Clifford groups appear naturally in the theory of topological error correction and abelian anyon models. We demonstrate a generalized Gottesman-Knill theorem, stating that every Clifford circuit can be efficiently classically simulated. We additionally provide circuits for a universal quantum computing scheme based on local two-qudit Clifford gates and magic states.


title: Towards multiloop local renormalization within Causal Loop-Tree Duality

abstract: Renormalization is a well-known technique to get rid of ultraviolet (UV) singularities. When relying on Dimensional Regularization (DREG), these become manifest as $\epsilon$-poles, allowing to define counter-terms with useful recursive properties. However, this procedure requires to work at \emph{integral-level} and poses difficulties to achieve a smooth combination with semi-numerical approaches. This article is devoted to the development of an integrand-level renormalization formalism, better suited for semi or fully numerical calculations. Starting from the Loop-Tree Duality (LTD), we keep the causal representations of the integrands of multiloop Feynman diagrams and explore their UV behaviour. Then, we propose a strategy that allows to build local counter-terms, capable of rendering the expressions integrable in the high-energy limit and in four space-time dimensions. Our procedure was tested on diagrams up to three-loops, and we found a remarkably smooth cancellation of divergences. The results of this work constitute a powerful step towards a fully local renormalization framework in QFT.


title: Binary dynamics to second post-Newtonian order in scalar-tensor and Einstein-scalar-Gauss-Bonnet gravity from effective field theory

abstract: Using effective field theory methods, we compute in detail the Lagrangian for the conservative dynamics of compact binary systems, for spinless constituents and in the gravitationally bound case, in massless scalar-tensor (ST) and Einstein-scalar-Gauss-Bonnet gravity (EsGB) to the second post-Newtonian (2PN) order. We employ the Kaluza-Klein parametrization of the metric, and demonstrate that, also in this case, a significant reduction in the number of diagrams is achieved. Still, to the 2PN accuracy, an additional 39 diagrams involving scalar interactions must be added to the pure general relativistic result. Diagrams involving the Gauss-Bonnet term at the formal 1PN order are computed and shown to scale as a 3PN correction when observational constraints are taken into account. We also investigate at which order the additional $\mathcal{O}(G^3)$ topologies contribute. Finally, the results derived in this paper provide an essential step toward the correct derivation of the conservative dynamics at the 3PN level in ST and EsGB theories.


title: The distribution of $\gcd(n,\phi(n))$

abstract: Let $\phi(n)$denote Euler's phi function. We study the distribution of the numbers $gcd(n,\phi(n))$ and their divisors. Our results generalize previous results of Erd\"{o}s and Pollack.


title: Small values and forbidden values for the Fourier antidiagonal constant of a finite group

abstract: For a finite group $G$, let ${\rm AD}(G)$ denote the Fourier norm of the antidiagonal in $G\times G$. It was shown recently by the author (IMRN, 2023) that ${\rm AD}(G)$ coincides with the amenability constant of the Fourier algebra of $G$, and is equal to the normalized sum of the cubes of character degrees of $G$. Motivated by a gap result for amenability constants due to Johnson (JLMS, 1994), we determine exactly which numbers in the interval $[1,2]$ arise as values of ${\rm AD}(G)$. As a by-product, we show that the set of values of ${\rm AD}(G)$ does not contain all its limit points. Some other calculations or bounds for ${\rm AD}(G)$ are given for familiar classes of finite groups. We also indicate a connection between ${\rm AD}(G)$ and the commuting probability of $G$, and use this to show that every finite group $G$ satisfying ${\rm AD}(G)< \frac{61}{15}$ must be solvable; here the value $\frac{61}{15}$ is best possible.


title: Asymptotics of Learning with Deep Structured (Random) Features

abstract: For a large class of feature maps we provide a tight asymptotic characterisation of the test error associated with learning the readout layer, in the high-dimensional limit where the input dimension, hidden layer widths, and number of training samples are proportionally large. This characterization is formulated in terms of the population covariance of the features. Our work is partially motivated by the problem of learning with Gaussian rainbow neural networks, namely deep non-linear fully-connected networks with random but structured weights, whose row-wise covariances are further allowed to depend on the weights of previous layers. For such networks we also derive a closed-form formula for the feature covariance in terms of the weight matrices. We further find that in some cases our results can capture feature maps learned by deep, finite-width neural networks trained under gradient descent.

categories: stat.ML cond-mat.dis-nn cs.LG math.ST stat.TH


title: Real-time 3D-aware Portrait Editing from a Single Image

abstract: This work presents 3DPE, a practical method that can efficiently edit a face image following given prompts, like reference images or text descriptions, in a 3D-aware manner. To this end, a lightweight module is distilled from a 3D portrait generator and a text-to-image model, which provide prior knowledge of face geometry and superior editing capability, respectively. Such a design brings two compelling advantages over existing approaches. First, our system achieves real-time editing with a feedforward network (i.e., ~0.04s per image), over 100x faster than the second competitor. Second, thanks to the powerful priors, our module could focus on the learning of editing-related variations, such that it manages to handle various types of editing simultaneously in the training phase and further supports fast adaptation to user-specified customized types of editing during inference (e.g., with ~5min fine-tuning per style). The code, the model, and the interface will be made publicly available to facilitate future research.


title: A formally motivated retrieval framework applied to the high resolution transmission spectrum of HD 189733 b

abstract: Ground-based high-resolution spectra provide a powerful tool for characterising exoplanet atmospheres. However, they are greatly hampered by the dominating telluric and stellar lines, which need to be removed prior to any analysis. Such removal techniques ("preparing pipelines") deform the spectrum, hence a key point is to account for this process in the forward models used in retrievals. We develop a formal derivation on how to prepare froward models for retrievals, in the case where the telluric and instrumental deformations can be represented as a matrix multiplied element-wise with the data. We also introduce the notion of "Bias Pipeline Metric" (BPM), that can be used to compare the bias potential of preparing pipelines. We use the resulting framework to retrieve simulated observations of 1-D and 3-D exoplanet atmospheres and to re-analyse high-resolution ($\mathcal{R} \approx 80\,400$) near infrared (0.96--1.71 $\mu$m) CARMENES transit data of HD~189733~b. We compare these results with those obtained from a CCF analysis. With our fiducial retrieval, we find a blueshift of the absorption features of $-5.51^{+0.66}_{-0.53}$ km$\cdot$s$^{-1}$. In addition, we retrieve a H$_2$O $\log_{10}$(VMR) of $-2.39^{+0.12}_{-0.16}$ and a temperature of $660^{+6}_{-11}$ K. We are also able to put upper limits for the abundances of CH$_4$, CO, H$_2$S, HCN and NH$_3$, consistent with a sub-solar metallicity atmosphere enriched in H$_2$O. We retrieve a broadened line shape, consistent with rotation- and wind-induced line broadening. Finally, we find a lower limit for the pressure of an opaque cloud consistent with a clear atmosphere, and find no evidence for hazes.


title: Hallucinations or Attention Misdirection? The Path to Strategic Value Extraction in Business Using Large Language Models

abstract: Large Language Models with transformer architecture have revolutionized the domain of text generation, setting unprecedented benchmarks. Despite their impressive capabilities, LLMs have been criticized for generating outcomes that deviate from factual accuracy or display logical inconsistencies, phenomena commonly referred to as hallucinations. This term, however, has often been misapplied to any results deviating from the instructor's expectations, which this paper defines as attention misdirection rather than true hallucinations. Understanding the distinction between hallucinations and attention misdirection becomes increasingly relevant in business contexts, where the ramifications of such errors can significantly impact the value extraction from these inherently pre-trained models. This paper highlights the best practices of the PGI, Persona, Grouping, and Intelligence, method, a strategic framework that achieved a remarkable error rate of only 3,15 percent across 4,000 responses generated by GPT in response to a real business challenge. It emphasizes that by equipping experimentation with knowledge, businesses can unlock opportunities for innovation through the use of these natively pre-trained models. This reinforces the notion that strategic application grounded in a skilled team can maximize the benefits of emergent technologies such as the LLMs.


title: Multidimensional Signaling with a Resource Constraint

abstract: We study multidimensional signaling (cognitive/non-cognitive) as a sender's portfolio choice with a resource constraint. We establish the existence of a unique monotone D1 equilibrium where the cognitive (non-cognitive) signal increases (decreases) in sender type and the sum of the two increases in sender type. The equilibrium is characterized by two threshold sender types. The low threshold is one where a kink occurs in signaling. The constraint is binding only for sender types above it. The high threshold is the other one, above which all types spend all the resources in cognitive signal with pooling and discontinuity on the top.


title: On a theorem of B. Keller on Yoneda algebras of simple modules

abstract: A theorem of Keller states that the Yoneda algebra of the simple modules over a finite-dimensional algebra is generated in cohomological degrees $0$ and $1$ as a minimal $A_\infty$-algebra. We provide a proof of an extension of Keller's theorem to abelian length categories by reducing the problem to a particular class of Nakayama algebras, where the claim can be shown by direct computation.


title: Information Elicitation in Agency Games

abstract: Rapid progress in scalable, commoditized tools for data collection and data processing has made it possible for firms and policymakers to employ ever more complex metrics as guides for decision-making. These developments have highlighted a prevailing challenge -- deciding *which* metrics to compute. In particular, a firm's ability to compute a wider range of existing metrics does not address the problem of *unknown unknowns*, which reflects informational limitations on the part of the firm. To guide the choice of metrics in the face of this informational problem, we turn to the evaluated agents themselves, who may have more information than a principal about how to measure outcomes effectively. We model this interaction as a simple agency game, where we ask: *When does an agent have an incentive to reveal the observability of a cost-correlated variable to the principal?* There are two effects: better information reduces the agent's information rents but also makes some projects go forward that otherwise would fail. We show that the agent prefers to reveal information that exposes a strong enough differentiation between high and low costs. Expanding the agent's action space to include the ability to *garble* their information, we show that the agent often prefers to garble over full revelation. Still, giving the agent the ability to garble can lead to higher total welfare. Our model has analogies with price discrimination, and we leverage some of these synergies to analyze total welfare.


title: Interplay between 2D ferromagnetism and transport at the surface of FeSi

abstract: FeSi is a curious example of a $d$-electron system that manifests many of the same phenomena associated with $f$-electron Kondo insulators, including conducting surface states with potentially non-trivial topology. Here we investigate the magnetization and magnetotransport of these surface states and how a 2D ferromagnetic state at the surface of FeSi influences the surface conductivity. We confirm the 2D ferromagnetism via a systematic study of magnetization on groups of filtered fragments with increasing surface area-to-volume ratios, identifying characteristic temperatures and magnetic fields associated with the ordered state. The paramagnetic to ferromagnetic transition appears broadened, suggesting disorder, which allows spin fluctuations to manifest up to at least 9 T at 2 K. This highlights the need to understand the relation between the disorder of the 2D ferromagnetism and the surface conductivity in FeSi.


title: Can Watermarks Survive Translation? On the Cross-lingual Consistency of Text Watermark for Large Language Models

abstract: Text watermarking technology aims to tag and identify content produced by large language models (LLMs) to prevent misuse. In this study, we introduce the concept of cross-lingual consistency in text watermarking, which assesses the ability of text watermarks to maintain their effectiveness after being translated into other languages. Preliminary empirical results from two LLMs and three watermarking methods reveal that current text watermarking technologies lack consistency when texts are translated into various languages. Based on this observation, we propose a Cross-lingual Watermark Removal Attack (CWRA) to bypass watermarking by first obtaining a response from an LLM in a pivot language, which is then translated into the target language. CWRA can effectively remove watermarks, decreasing the AUCs to a random-guessing level without performance loss. Furthermore, we analyze two key factors that contribute to the cross-lingual consistency in text watermarking and propose X-SIR as a defense method against CWRA. Code: https://github.com/zwhe99/X-SIR.


title: OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems

abstract: Recent advancements have seen Large Language Models (LLMs) and Large Multimodal Models (LMMs) surpassing general human capabilities in various tasks, approaching the proficiency level of human experts across multiple domains. With traditional benchmarks becoming less challenging for these models, new rigorous challenges are essential to gauge their advanced abilities. In this work, we present OlympiadBench, an Olympiad-level bilingual multimodal scientific benchmark, featuring 8,476 problems from Olympiad-level mathematics and physics competitions, including the Chinese college entrance exam. Each problem is detailed with expert-level annotations for step-by-step reasoning. Evaluating top-tier models on OlympiadBench, we implement a comprehensive assessment methodology to accurately evaluate model responses. Notably, the best-performing model, GPT-4V, attains an average score of 17.97% on OlympiadBench, with a mere 10.74% in physics, highlighting the benchmark rigor and the intricacy of physical reasoning. Our analysis orienting GPT-4V points out prevalent issues with hallucinations, knowledge omissions, and logical fallacies. We hope that our challenging benchmark can serve as a valuable resource for helping future AGI research endeavors. The data and evaluation code are available at \url{https://github.com/OpenBMB/OlympiadBench}


title: Geometry-Informed Neural Networks

abstract: Geometry is a ubiquitous language of computer graphics, design, and engineering. However, the lack of large shape datasets limits the application of state-of-the-art supervised learning methods and motivates the exploration of alternative learning strategies. To this end, we introduce geometry-informed neural networks (GINNs) to train shape generative models \emph{without any data}. GINNs combine (i) learning under constraints, (ii) neural fields as a suitable representation, and (iii) generating diverse solutions to under-determined problems. We apply GINNs to several two and three-dimensional problems of increasing levels of complexity. Our results demonstrate the feasibility of training shape generative models in a data-free setting. This new paradigm opens several exciting research directions, expanding the application of generative models into domains where data is sparse.


title: Two photons everywhere

abstract: We discuss two-photon physics, taking for illustration the particular but topical case of resonance fluorescence. We show that the basic concepts of interferences and correlations provide at the two-photon level an independent and drastically different picture than at the one-photon level, with landscapes of correlations that reveal various processes by spanning over all the possible frequencies at which the system can emit. Such landscapes typically present lines of photon bunching and circles of antibunching. The theoretical edifice to account for these features rests on two pillars: i) a theory of frequency-resolved photon correlations and ii) admixing classical and quantum fields. While experimental efforts have been to date concentrated on correlations between spectral peaks, strong correlations exist between photons emitted away from the peaks, which are accessible only through multiphoton observables. These could be exploited for both fundamental understanding of quantum-optical processes as well as applications by harnessing these unsuspected resources.


title: Spectral mod p Satake isomorphism for GL_n

abstract: Let $K/\mathbb{Q}_p$ be a finite extension with residue field $k$. By a work of Emerton--Gee, irreducible components inside the reduced special fiber of the moduli stack of rank $n$ \'etale $(\varphi,\Gamma)$-modules are labeled by Serre weights of $\mathrm{GL}_n(k)$. Let $\sigma$ be a non-Steinberg Serre weight and $\mathcal{C}_\sigma$ be the corresponding irreducible component. Motivated by the categorical $p$-adic local Langlands program, we construct a natural injective map $\mathcal{O}(\mathcal{C}_\sigma) \hookrightarrow \mathcal{H}(\sigma)$ from the ring of global functions on $\mathcal{C}_\sigma$ to the Hecke algebra of $\sigma$ compatible with the mod $p$ Satake isomorphism by Herzig and Henniart--Vign\'eras in a suitable sense. For sufficiently generic $\sigma$, we prove that it is an isomorphism. As an application, we obtain a natural stratification of the irreducible component whose strata are equipped with a parabolic structure. Our main input is a construction of a morphism from an integral Hecke algebra of a generic tame type to the ring of global functions on a tamely potentially crystalline Emerton--Gee stack.


title: Chasing Convex Functions with Long-term Constraints

abstract: We introduce and study a family of online metric problems with long-term constraints. In these problems, an online player makes decisions $\mathbf{x}_t$ in a metric space $(X,d)$ to simultaneously minimize their hitting cost $f_t(\mathbf{x}_t)$ and switching cost as determined by the metric. Over the time horizon $T$, the player must satisfy a long-term demand constraint $\sum_{t} c(\mathbf{x}_t) \geq 1$, where $c(\mathbf{x}_t)$ denotes the fraction of demand satisfied at time $t$. Such problems can find a wide array of applications to online resource allocation in sustainable energy and computing systems. We devise optimal competitive and learning-augmented algorithms for specific instantiations of these problems, and further show that our proposed algorithms perform well in numerical experiments.


title: Misalignment, Learning, and Ranking: Harnessing Users Limited Attention

abstract: In digital health and EdTech, recommendation systems face a significant challenge: users often choose impulsively, in ways that conflict with the platform's long-term payoffs. This misalignment makes it difficult to effectively learn to rank items, as it may hinder exploration of items with greater long-term payoffs. Our paper tackles this issue by utilizing users' limited attention spans. We propose a model where a platform presents items with unknown payoffs to the platform in a ranked list to $T$ users over time. Each user selects an item by first considering a prefix window of these ranked items and then picking the highest preferred item in that window (and the platform observes its payoff for this item). We study the design of online bandit algorithms that obtain vanishing regret against hindsight optimal benchmarks. We first consider adversarial window sizes and stochastic iid payoffs. We design an active-elimination-based algorithm that achieves an optimal instance-dependent regret bound of $O(\log(T))$, by showing matching regret upper and lower bounds. The key idea is using the combinatorial structure of the problem to either obtain a large payoff from each item or to explore by getting a sample from that item. This method systematically narrows down the item choices to enhance learning efficiency and payoff. Second, we consider adversarial payoffs and stochastic iid window sizes. We start from the full-information problem of finding the permutation that maximizes the expected payoff. By a novel combinatorial argument, we characterize the polytope of admissible item selection probabilities by a permutation and show it has a polynomial-size representation. Using this representation, we show how standard algorithms for adversarial online linear optimization in the space of admissible probabilities can be used to obtain a polynomial-time algorithm with $O(\sqrt{T})$ regret.


title: JWST MIRI Imager Observations of Supernova SN 1987A

abstract: There exist very few mid-infrared (IR) observations of supernovae (SNe) in general. Therefore, SN 1987A, the closest visible SN in 400 years, gives us the opportunity to explore the mid-IR properties of SNe, the dust in their ejecta and surrounding medium, and to witness the birth of a SN remnant (SNR). The James Webb Space Telescope (JWST), with its high spatial resolution and extreme sensitivity, gives a new view on these issues. We report on the first imaging observations obtained with the Mid-InfraRed Instrument (MIRI). We build temperature maps and discuss the morphology of the nascent SNR. Our results show that the temperatures in the equatorial ring (ER) are quite non-uniform. This could be due to dust destruction in some parts of the ring, as had been assumed in some previous works. We show that the IR emission extends beyond the ER, illustrating the fact that the shock wave has now passed through this ring to affect the circumstellar medium on a larger scale. Finally, while sub-mm Atacama Large Millimeter Array (ALMA) observations have hinted at the location of the compact remnant of SN 1987A, we note that our MIRI data have found no such evidence.


title: Corrective Machine Unlearning

abstract: Machine Learning models increasingly face data integrity challenges due to the use of large-scale training datasets drawn from the internet. We study what model developers can do if they detect that some data was manipulated or incorrect. Such manipulated data can cause adverse effects like vulnerability to backdoored samples, systematic biases, and in general, reduced accuracy on certain input domains. Often, all manipulated training samples are not known, and only a small, representative subset of the affected data is flagged. We formalize "Corrective Machine Unlearning" as the problem of mitigating the impact of data affected by unknown manipulations on a trained model, possibly knowing only a subset of impacted samples. We demonstrate that the problem of corrective unlearning has significantly different requirements from traditional privacy-oriented unlearning. We find most existing unlearning methods, including the gold-standard retraining-from-scratch, require most of the manipulated data to be identified for effective corrective unlearning. However, one approach, SSD, achieves limited success in unlearning adverse effects with just a small portion of the manipulated samples, showing the tractability of this setting. We hope our work spurs research towards developing better methods for corrective unlearning and offers practitioners a new strategy to handle data integrity challenges arising from web-scale training.


title: Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment

abstract: Large Language Models (LLMs) are powerful zero-shot assessors and are increasingly used in real-world situations such as for written exams or benchmarking systems. Despite this, no existing work has analyzed the vulnerability of judge-LLMs against adversaries attempting to manipulate outputs. This work presents the first study on the adversarial robustness of assessment LLMs, where we search for short universal phrases that when appended to texts can deceive LLMs to provide high assessment scores. Experiments on SummEval and TopicalChat demonstrate that both LLM-scoring and pairwise LLM-comparative assessment are vulnerable to simple concatenation attacks, where in particular LLM-scoring is very susceptible and can yield maximum assessment scores irrespective of the input text quality. Interestingly, such attacks are transferable and phrases learned on smaller open-source LLMs can be applied to larger closed-source models, such as GPT3.5. This highlights the pervasive nature of the adversarial vulnerabilities across different judge-LLM sizes, families and methods. Our findings raise significant concerns on the reliability of LLMs-as-a-judge methods, and underscore the importance of addressing vulnerabilities in LLM assessment methods before deployment in high-stakes real-world scenarios.


title: D-Flow: Differentiating through Flows for Controlled Generation

abstract: Taming the generation outcome of state of the art Diffusion and Flow-Matching (FM) models without having to re-train a task-specific model unlocks a powerful tool for solving inverse problems, conditional generation, and controlled generation in general. In this work we introduce D-Flow, a simple framework for controlling the generation process by differentiating through the flow, optimizing for the source (noise) point. We motivate this framework by our key observation stating that for Diffusion/FM models trained with Gaussian probability paths, differentiating through the generation process projects gradient on the data manifold, implicitly injecting the prior into the optimization process. We validate our framework on linear and non-linear controlled generation problems including: image and audio inverse problems and conditional molecule generation reaching state of the art performance across all.


title: Performance Evaluation and Analysis of Thresholding-based Interference Mitigation for Automotive Radar Systems

abstract: In automotive radar, time-domain thresholding (TD-TH) and time-frequency domain thresholding (TFD-TH) are crucial techniques underpinning numerous interference mitigation methods. Despite their importance, comprehensive evaluations of these methods in dense traffic scenarios with different types of interference are limited. In this study, we segment automotive radar interference into three distinct categories. Utilizing the in-house traffic scenario and automotive radar simulator, we evaluate interference mitigation methods across multiple metrics: probability of detection, signal-to-interference-plus-noise ratio, and phase error involving hundreds of targets and dozens of interfering radars. The numerical results highlight that TFD-TH is more effective than TD-TH, particularly as the density and signal correlation of interfering radars escalate.


title: Maxentropy completion and properties of some partially defined Stationary Markov chains

abstract: We consider a stationary Markovian evolution with values on a disjointly partitioned set space $I\sqcup {\cal E}$. The evolution is visible (in the sense of knowing the transition probabilities) on the states in $I$ but not for the states in ${\cal E}$. One only knows some partial information on the transition probabilities on ${\cal E}$, the input and output transition probabilities and some constraints of the transition probabilities on ${\cal E}$. Under some conditions we supply the transition probabilities on ${\cal E}$ that satisfies the maximum entropy principle.


title: Coercing LLMs to do and reveal (almost) anything

abstract: It has recently been shown that adversarial attacks on large language models (LLMs) can "jailbreak" the model into making harmful statements. In this work, we argue that the spectrum of adversarial attacks on LLMs is much larger than merely jailbreaking. We provide a broad overview of possible attack surfaces and attack goals. Based on a series of concrete examples, we discuss, categorize and systematize attacks that coerce varied unintended behaviors, such as misdirection, model control, denial-of-service, or data extraction. We analyze these attacks in controlled experiments, and find that many of them stem from the practice of pre-training LLMs with coding capabilities, as well as the continued existence of strange "glitch" tokens in common LLM vocabularies that should be removed for security reasons.


title: Betting on what is neither verifiable nor falsifiable

abstract: Prediction markets are useful for estimating probabilities of claims whose truth will be revealed at some fixed time -- this includes questions about the values of real-world events (i.e. statistical uncertainty), and questions about the values of primitive recursive functions (i.e. logical or algorithmic uncertainty). However, they cannot be directly applied to questions without a fixed resolution criterion, and real-world applications of prediction markets to such questions often amount to predicting not whether a sentence is true, but whether it will be proven. Such questions could be represented by countable unions or intersections of more basic events, or as First-Order-Logic sentences on the Arithmetical Hierarchy (or even beyond FOL, as hyperarithmetical sentences). In this paper, we propose an approach to betting on such events via options, or equivalently as bets on the outcome of a "verification-falsification game". Our work thus acts as an alternative to the existing framework of Garrabrant induction for logical uncertainty, and relates to the stance known as constructivism in the philosophy of mathematics; furthermore it has broader implications for philosophy and mathematical logic.


title: Statistical validation of a deep learning algorithm for dental anomaly detection in intraoral radiographs using paired data

abstract: This article describes the clinical validation study setup, statistical analysis and results for a deep learning algorithm which detects dental anomalies in intraoral radiographic images, more specifically caries, apical lesions, root canal treatment defects, marginal defects at crown restorations, periodontal bone loss and calculus. The study compares the detection performance of dentists using the deep learning algorithm to the prior performance of these dentists evaluating the images without algorithmic assistance. Calculating the marginal profit and loss of performance from the annotated paired image data allows for a quantification of the hypothesized change in sensitivity and specificity. The statistical significance of these results is extensively proven using both McNemar's test and the binomial hypothesis test. The average sensitivity increases from $60.7\%$ to $85.9\%$, while the average specificity slightly decreases from $94.5\%$ to $92.7\%$. We prove that the increase of the area under the localization ROC curve (AUC) is significant (from $0.60$ to $0.86$ on average), while the average AUC is bounded by the $95\%$ confidence intervals ${[}0.54, 0.65{]}$ and ${[}0.82, 0.90{]}$. When using the deep learning algorithm for diagnostic guidance, the dentist can be $95\%$ confident that the average true population sensitivity is bounded by the range $79.6\%$ to $91.9\%$. The proposed paired data setup and statistical analysis can be used as a blueprint to thoroughly test the effect of a modality change, like a deep learning based detection and/or segmentation, on radiographic images.


title: 25-Fold Resolution Enhancement of X-ray Microscopy Using Multipixel Ghost Imaging

abstract: Hard x-ray imaging is indispensable across diverse fields owing to its high penetrability. However, the resolution of traditional x-ray imaging modalities, such as computed tomography (CT) systems, is constrained by factors including beam properties, the absence of optical components, and detection resolution. As a result, typical resolution in commercial imaging systems is limited to a few hundred microns. This study advances high-photon-energy imaging by extending the concept of computational ghost imaging to multipixel ghost imaging with x-rays. We demonstrate a remarkable enhancement in resolution from 500 microns to approximately 20 microns for an image spanning 0.9 by 1 cm^2, comprised of 400,000 pixels and involving only 1000 realizations. Furthermore, we present a high-resolution CT reconstruction using our method, revealing enhanced visibility and resolution. Our achievement is facilitated by an innovative x-ray lithography technique and the computed tiling of images captured by each detector pixel. Importantly, this method can be scaled up for larger images without sacrificing the short measurement time, thereby opening intriguing possibilities for noninvasive high-resolution imaging of small features that are invisible with the present modalities.


title: Patterns in trees and quantum automorphism groups

abstract: We prove that given a fixed finite tree $P$, almost all trees contain $P$ as a subtree. Moreover, the inclusion can be made so that it induces an embedding of the corresponding (quantum) automorphism groups, thereby providing generic properties of the latter.


title: Spectral Efficiency Maximization for Active RIS-aided Cell-Free Massive MIMO Systems with Imperfect CSI

abstract: A cell-free network merged with active reconfigurable reflecting surfaces (RIS) is investigated in this paper. Based on the imperfect channel state information (CSI), the aggregated channel from the user to the access point (AP) is initially estimated using the linear minimum mean square error (LMMSE) technique. The central processing unit (CPU) then detects uplink data from individual users through the utilization of the maximum ratio combining (MRC) approach, relying on the estimated channel. Then, a closed-form expression for uplink spectral efficiency (SE) is derived which demonstrates its reliance on statistical CSI (S-CSI) alone. The amplitude gain of each active RIS element is derived in a closed-form expression as a function of the number of active RIS elements, the number of users, and the size of each reflecting element. A soft actor-critic (SAC) algorithm is utilized to design the phase shift of the active RIS to maximize the uplink SE. Simulation results emphasize the robustness of the proposed SAC algorithm, showcasing its effectiveness in cell-free networks under the influence of imperfect CSI.


title: Probability Tools for Sequential Random Projection

abstract: We introduce the first probabilistic framework tailored for sequential random projection, an approach rooted in the challenges of sequential decision-making under uncertainty. The analysis is complicated by the sequential dependence and high-dimensional nature of random variables, a byproduct of the adaptive mechanisms inherent in sequential decision processes. Our work features a novel construction of a stopped process, facilitating the analysis of a sequence of concentration events that are interconnected in a sequential manner. By employing the method of mixtures within a self-normalized process, derived from the stopped process, we achieve a desired non-asymptotic probability bound. This bound represents a non-trivial martingale extension of the Johnson-Lindenstrauss (JL) lemma, marking a pioneering contribution to the literature on random projection and sequential analysis.

categories: math.ST cs.DS cs.IT cs.NA math.IT math.NA math.PR stat.ML stat.TH


title: Learning causation event conjunction sequences

abstract: This is an examination of some methods that learn causations in event sequences. A causation is defined as a conjunction of one or more cause events occurring in an arbitrary order, with possible intervening non-causal events, that lead to an effect. The methods include recurrent and non-recurrent artificial neural networks (ANNs), as well as a histogram-based algorithm. An attention recurrent ANN performed the best of the ANNs, while the histogram algorithm was significantly superior to all the ANNs.


title: A mathematical model for wind-generated particle-fluid flow fields with an application to the helicopter cloud problem

abstract: We develop a model for the interaction of a fluid flowing above an otherwise static particle bed, with generally the particles being entrained or detrained into the fluid from the upper surface of the particle bed, and thereby forming a fully two phase fluidized cloud above the particle bed. The flow in this large scale fluidized region is treated as a two-phase flow, whilst the key processes of entrainment and detrainment from the particle bed are treated by examining the local dynamical force balances on the particles in a thin transition layer at the interface between the fully fluidized region and the static particle bed. This detailed consideration leads to the formation of an additional macroscopic boundary condition at this interface, which closes the two phase flow problem in the bulk fluidized region above. We then introduce an elementary model of the well known helicopter brownout problem, and use the theory developed in the first part of the paper to fully analyse this model, both analytically and numerically.


title: Partial Search in a Frozen Network is Enough to Find a Strong Lottery Ticket

abstract: Randomly initialized dense networks contain subnetworks that achieve high accuracy without weight learning -- strong lottery tickets (SLTs). Recently, Gadhikar et al. (2023) demonstrated that SLTs can also be found within a randomly pruned source network, thus reducing the SLT search space. However, this limits the search to SLTs that are even sparser than the source, leading to worse accuracy due to unintentionally high sparsity. This paper proposes a method that reduces the SLT search space by an arbitrary ratio independent of the desired SLT sparsity. A random subset of the initial weights is excluded from the search space by freezing it -- i.e., by either permanently pruning them or locking them as a fixed part of the SLT. In addition to reducing search space, the proposed random freezing can also provide the benefit of reducing the model size for inference. Furthermore, experimental results show that the proposed method finds SLTs with better accuracy-to-model size trade-off than the SLTs obtained from dense or randomly pruned source networks. In particular, the SLTs found in Frozen ResNets on image classification using ImageNet significantly improve the accuracy-to-search space and accuracy-to-model size trade-offs over SLTs within dense (non-freezing) or sparse (non-locking) random networks.


title: On the initial spin period distribution of neutron stars

abstract: We derive the initial spin period distribution of neutron stars by studying the population of young pulsars associated with supernova remnants. Our hierarchical Bayesian approach accounts for the measurement uncertainties of individual observations and selection effects. Without correcting for selection effects, as done in previous studies, we find that pulsar initial spin periods follow a Weibull distribution, peaking at 40~ms, which is favoured against the log-normal distribution with a Bayes factor of 200. The known selection effects in radio pulsar surveys, including pulse broadening and a period-dependent beaming fraction, have been quantitatively investigated. We show that, based on measurements of pulsar luminosity and spin period from the ATNF Pulsar Catalogue, the impact of pulse broadening on the inference of pulsar period distribution is likely to be insignificant. Correcting for the beaming selection effect, a Weibull distribution remains to be the preferred model, while its peak slightly shifts to longer periods at 50~ms. Our method will prove useful in constraining the birth properties of neutron stars in the Square Kilometre Array era.


title: Autoencoder with Ordered Variance for Nonlinear Model Identification

abstract: This paper presents a novel autoencoder with ordered variance (AEO) in which the loss function is modified with a variance regularization term to enforce order in the latent space. Further, the autoencoder is modified using ResNets, which results in a ResNet AEO (RAEO). The paper also illustrates the effectiveness of AEO and RAEO in extracting nonlinear relationships among input variables in an unsupervised setting.


title: Recent advances in chiral EFT based nuclear forces and their applications

abstract: During the past two decades, chiral effective field theory has evolved into a powerful tool to derive nuclear forces from first principles. Nearly all two-nucleon interactions have been worked out up to sixth order of chiral perturbation theory, while, with few exceptions, three-nucleon forces, which play a subtle, but crucial role in microscopic nuclear structure calculations, have been derived up to fifth order. We review the current status of these forces as well as their applications in nuclear many-body systems. While the ab initio description of light nuclei is generally very successful, we point out and analyze problems encountered with medium-mass nuclei. We also survey the construction of equations of state for symmetric nuclear matter and neutron-rich matter based on chiral forces. A focal point is the symmetry energy and its impact on neutron skins and systems of astrophysical relevance. The physics of neutron-rich systems, from nuclei to compact stars, is essentially determined by the density dependence of the symmetry energy. We review the status of predictions in comparison with latest empirical constraints, with particular attention to those extracted from parity violating electron scattering.


title: VN Network: Embedding Newly Emerging Entities with Virtual Neighbors

abstract: Embedding entities and relations into continuous vector spaces has attracted a surge of interest in recent years. Most embedding methods assume that all test entities are available during training, which makes it time-consuming to retrain embeddings for newly emerging entities. To address this issue, recent works apply the graph neural network on the existing neighbors of the unseen entities. In this paper, we propose a novel framework, namely Virtual Neighbor (VN) network, to address three key challenges. Firstly, to reduce the neighbor sparsity problem, we introduce the concept of the virtual neighbors inferred by rules. And we assign soft labels to these neighbors by solving a rule-constrained problem, rather than simply regarding them as unquestionably true. Secondly, many existing methods only use one-hop or two-hop neighbors for aggregation and ignore the distant information that may be helpful. Instead, we identify both logic and symmetric path rules to capture complex patterns. Finally, instead of one-time injection of rules, we employ an iterative learning scheme between the embedding method and virtual neighbor prediction to capture the interactions within. Experimental results on two knowledge graph completion tasks demonstrate that our VN network significantly outperforms state-of-the-art baselines. Furthermore, results on Subject/Object-R show that our proposed VN network is highly robust to the neighbor sparsity problem.


title: AgentScope: A Flexible yet Robust Multi-Agent Platform

abstract: With the rapid advancement of Large Language Models (LLMs), significant progress has been made in multi-agent applications. However, the complexities in coordinating agents' cooperation and LLMs' erratic performance pose notable challenges in developing robust and efficient multi-agent applications. To tackle these challenges, we propose AgentScope, a developer-centric multi-agent platform with message exchange as its core communication mechanism. The abundant syntactic tools, built-in agents and service functions, user-friendly interfaces for application demonstration and utility monitor, zero-code programming workstation, and automatic prompt tuning mechanism significantly lower the barriers to both development and deployment. Towards robust and flexible multi-agent application, AgentScope provides both built-in and customizable fault tolerance mechanisms. At the same time, it is also armed with system-level support for managing and utilizing multi-modal data, tools, and external knowledge. Additionally, we design an actor-based distribution framework, enabling easy conversion between local and distributed deployments and automatic parallel optimization without extra effort. With these features, AgentScope empowers developers to build applications that fully realize the potential of intelligent agents. We have released AgentScope at https://github.com/modelscope/agentscope, and hope AgentScope invites wider participation and innovation in this fast-moving field.


title: Wisdom of Committee: Distilling from Foundation Model to Specialized Application Model

abstract: Recent advancements in foundation models have yielded impressive performance across a wide range of tasks. Meanwhile, for specific applications, practitioners have been developing specialized application models. To enjoy the benefits of both kinds of models, one natural path is to transfer the knowledge in foundation models into specialized application models, which are generally more efficient for serving. Techniques from knowledge distillation may be applied here, where the application model learns to mimic the foundation model. However, specialized application models and foundation models have substantial gaps in capacity, employing distinct architectures, using different input features from different modalities, and being optimized on different distributions. These differences in model characteristics lead to significant challenges for distillation methods. In this work, we propose creating a teaching committee comprising both foundation model teachers and complementary teachers. Complementary teachers possess model characteristics akin to the student's, aiming to bridge the gap between the foundation model and specialized application models for a smoother knowledge transfer. Further, to accommodate the dissimilarity among the teachers in the committee, we introduce DiverseDistill, which allows the student to understand the expertise of each teacher and extract task knowledge. Our evaluations demonstrate that adding complementary teachers enhances student performance. Finally, DiverseDistill consistently outperforms baseline distillation methods, regardless of the teacher choices, resulting in significantly improved student performance.


title: Quantum Annealing and Graph Neural Networks for Solving TSP with QUBO

abstract: This paper explores the application of Quadratic Unconstrained Binary Optimization (QUBO) models in solving the Travelling Salesman Problem (TSP) through Quantum Annealing algorithms and Graph Neural Networks. Quantum Annealing (QA), a quantum-inspired optimization method that exploits quantum tunneling to escape local minima, is used to solve QUBO formulations of TSP instances on Coherent Ising Machines (CIMs). The paper also presents a novel approach where QUBO is employed as a loss function within a GNN architecture tailored for solving TSP efficiently. By leveraging GNN's capability to learn graph representations, this method finds approximate solutions to TSP with improved computational time compared to traditional exact solvers. The paper details how to construct a QUBO model for TSP by encoding city visits into binary variables and formulating constraints that guarantee valid tours. It further discusses the implementation of QUBO-based Quantum Annealing algorithm for TSP (QQA-TSP) and its feasibility demonstration using quantum simulation platforms. In addition, it introduces a Graph Neural Network solution for TSP (QGNN-TSP), which learns the underlying structure of the problem and produces competitive solutions via gradient descent over a QUBO-based loss function. The experimental results compare the performance of QQA-TSP against state-of-the-art classical solvers such as dynamic programming, Concorde, and Gurobi, while also presenting empirical outcomes from training and evaluating QGNN-TSP on various TSP datasets. The study highlights the promise of combining deep learning techniques with quantum-inspired optimization methods for solving NP-hard problems like TSP, suggesting future directions for enhancing GNN architectures and applying QUBO frameworks to more complex combinatorial optimization tasks.