title: Learning to Reduce: Optimal Representations of Structured Data in Prompting Large Language Models

abstract: Large Language Models (LLMs) have been widely used as general-purpose AI agents showing comparable performance on many downstream tasks. However, existing work shows that it is challenging for LLMs to integrate structured data (e.g. KG, tables, DBs) into their prompts; LLMs need to either understand long text data or select the most relevant evidence prior to inference, and both approaches are not trivial. In this paper, we propose a framework, Learning to Reduce, that fine-tunes a language model to generate a reduced version of an input context, given a task description and context input. The model learns to reduce the input context using On-Policy Reinforcement Learning and aims to improve the reasoning performance of a fixed LLM. Experimental results illustrate that our model not only achieves comparable accuracies in selecting the relevant evidence from an input context, but also shows generalizability on different datasets. We further show that our model helps improve the LLM's performance on downstream tasks especially when the context is long.


title: Mip-Grid: Anti-aliased Grid Representations for Neural Radiance Fields

abstract: Despite the remarkable achievements of neural radiance fields (NeRF) in representing 3D scenes and generating novel view images, the aliasing issue, rendering "jaggies" or "blurry" images at varying camera distances, remains unresolved in most existing approaches. The recently proposed mip-NeRF has addressed this challenge by rendering conical frustums instead of rays. However, it relies on MLP architecture to represent the radiance fields, missing out on the fast training speed offered by the latest grid-based methods. In this work, we present mip-Grid, a novel approach that integrates anti-aliasing techniques into grid-based representations for radiance fields, mitigating the aliasing artifacts while enjoying fast training time. The proposed method generates multi-scale grids by applying simple convolution operations over a shared grid representation and uses the scale-aware coordinate to retrieve features at different scales from the generated multi-scale grids. To test the effectiveness, we integrated the proposed method into the two recent representative grid-based methods, TensoRF and K-Planes. Experimental results demonstrate that mip-Grid greatly improves the rendering performance of both methods and even outperforms mip-NeRF on multi-scale datasets while achieving significantly faster training time. For code and demo videos, please see https://stnamjef.github.io/mipgrid.github.io/.


title: Any two-coloring of the plane contains monochromatic 3-term arithmetic progressions

abstract: A conjecture of Erd\H{o}s, Graham, Montgomery, Rothschild, Spencer and Straus states that, with the exception of equilateral triangles, any two-coloring of the plane will have a monochromatic congruent copy of every three-point configuration. This conjecture is known only for special classes of configurations. In this manuscript, we confirm one of the most natural open cases; that is, every two-coloring of the plane admits a monochromatic congruent copy of any $3$-term arithmetic progression.


title: Tight Inapproximability of Nash Equilibria in Public Goods Games

abstract: We study public goods games, a type of game where every player has to decide whether or not to produce a good which is public, i.e., neighboring players can also benefit from it. Specifically, we consider a setting where the good is indivisible and where the neighborhood structure is represented by a directed graph, with the players being the nodes. Papadimitriou and Peng (2023) recently showed that in this setting computing mixed Nash equilibria is PPAD-hard, and that this remains the case even for $\varepsilon$-well-supported approximate equilibria for some sufficiently small constant $\varepsilon$. In this work, we strengthen this inapproximability result by showing that the problem remains PPAD-hard for any non-trivial approximation parameter $\varepsilon$.


title: Center-of-mass recoil effects on the annihilation and formation of dark matter bound pairs

abstract: For a quantitative investigation on the time evolution of heavy thermal dark matter at and after thermal freeze-out, near-threshold processes need to be taken into account which have a large impact on the observed dark matter relic abundance. In this conference paper, we study the recoil effect of heavy dark matter pairs in a thermal bath and compute the annihilation cross section and the decay width as well as the bound-state formation cross section of dark matter fermion-antifermion pairs in the laboratory frame within the framework of potential non-relativistic effective field theories at finite temperature. For the considered hierarchy of energy scales, we highlight the effect of the recoil corrections to the thermal rates.


title: Towards Understanding Counseling Conversations: Domain Knowledge and Large Language Models

abstract: Understanding the dynamics of counseling conversations is an important task, yet it is a challenging NLP problem regardless of the recent advance of Transformer-based pre-trained language models. This paper proposes a systematic approach to examine the efficacy of domain knowledge and large language models (LLMs) in better representing conversations between a crisis counselor and a help seeker. We empirically show that state-of-the-art language models such as Transformer-based models and GPT models fail to predict the conversation outcome. To provide richer context to conversations, we incorporate human-annotated domain knowledge and LLM-generated features; simple integration of domain knowledge and LLM features improves the model performance by approximately 15%. We argue that both domain knowledge and LLM-generated features can be exploited to better characterize counseling conversations when they are used as an additional context to conversations.


title: Random-Order Online Independent Set of Intervals and Hyperrectangles

abstract: In the Maximum Independent Set of Hyperrectangles problem, we are given a set of $n$ (possibly overlapping) $d$-dimensional axis-aligned hyperrectangles, and the goal is to find a subset of non-overlapping hyperrectangles of maximum cardinality. For $d=1$, this corresponds to the classical Interval Scheduling problem, where a simple greedy algorithm returns an optimal solution. In the offline setting, for $d$-dimensional hyperrectangles, polynomial time $(\log n)^{O(d)}$-approximation algorithms are known. However, the problem becomes notably challenging in the online setting, where the input objects (hyperrectangles) appear one by one in an adversarial order, and on the arrival of an object, the algorithm needs to make an immediate and irrevocable decision whether or not to select the object while maintaining the feasibility. Even for interval scheduling, an $\Omega(n)$ lower bound is known on the competitive ratio. To circumvent these negative results, in this work, we study the online maximum independent set of axis-aligned hyperrectangles in the random-order arrival model, where the adversary specifies the set of input objects which then arrive in a uniformly random order. Starting from the prototypical secretary problem, the random-order model has received significant attention to study algorithms beyond the worst-case competitive analysis. Surprisingly, we show that the problem in the random-order model almost matches the best-known offline approximation guarantees, up to polylogarithmic factors. In particular, we give a simple $(\log n)^{O(d)}$-competitive algorithm for $d$-dimensional hyperrectangles in this model, which runs in $\tilde{O_d}(n)$ time. Our approach also yields $(\log n)^{O(d)}$-competitive algorithms in the random-order model for more general objects such as $d$-dimensional fat objects and ellipsoids. Furthermore, our guarantees hold with high probability.


title: Comparing Graph Transformers via Positional Encodings

abstract: The distinguishing power of graph transformers is closely tied to the choice of positional encoding: features used to augment the base transformer with information about the graph. There are two primary types of positional encoding: absolute positional encodings (APEs) and relative positional encodings (RPEs). APEs assign features to each node and are given as input to the transformer. RPEs instead assign a feature to each pair of nodes, e.g., graph distance, and are used to augment the attention block. A priori, it is unclear which method is better for maximizing the power of the resulting graph transformer. In this paper, we aim to understand the relationship between these different types of positional encodings. Interestingly, we show that graph transformers using APEs and RPEs are equivalent in terms of distinguishing power. In particular, we demonstrate how to interchange APEs and RPEs while maintaining their distinguishing power in terms of graph transformers. Based on our theoretical results, we provide a study on several APEs and RPEs (including the resistance distance and the recently introduced stable and expressive positional encoding (SPE)) and compare their distinguishing power in terms of transformers. We believe our work will help navigate the huge number of choices of positional encoding and will provide guidance on the future design of positional encodings for graph transformers.


title: An Exploratory Analysis of COVID Bot vs Human Disinformation Dissemination stemming from the Disinformation Dozen on Telegram

abstract: The COVID-19 pandemic of 2021 led to a worldwide health crisis that was accompanied by an infodemic. A group of 12 social media personalities, dubbed the ``Disinformation Dozen", were identified as key in spreading disinformation regarding the COVID-19 virus, treatments, and vaccines. This study focuses on the spread of disinformation propagated by this group on Telegram, a mobile messaging and social media platform. After segregating users into three groups -- the Disinformation Dozen, bots, and humans --, we perform an investigation with a dataset of Telegram messages from January to June 2023, comparatively analyzing temporal, topical, and network features. We observe that the Disinformation Dozen are highly involved in the initial dissemination of disinformation but are not the main drivers of the propagation of disinformation. Bot users are extremely active in conversation threads, while human users are active propagators of information, disseminating posts between Telegram channels through the forwarding mechanism.


title: A sufficient condition for the height function to be constant in $ I_g\times_\rho \mathbb{P}^n $

abstract: This paper makes some modifications to the warped product space. Based on Alias,Impera and Rigoli, a warping function is added to the warped product space. This new function affects the Riemannian metric of the warped product space. In this new warped product space, we continue to discuss the sufficient condition for calculating the height of the immersed surface.


title: Compression Robust Synthetic Speech Detection Using Patched Spectrogram Transformer

abstract: Many deep learning synthetic speech generation tools are readily available. The use of synthetic speech has caused financial fraud, impersonation of people, and misinformation to spread. For this reason forensic methods that can detect synthetic speech have been proposed. Existing methods often overfit on one dataset and their performance reduces substantially in practical scenarios such as detecting synthetic speech shared on social platforms. In this paper we propose, Patched Spectrogram Synthetic Speech Detection Transformer (PS3DT), a synthetic speech detector that converts a time domain speech signal to a mel-spectrogram and processes it in patches using a transformer neural network. We evaluate the detection performance of PS3DT on ASVspoof2019 dataset. Our experiments show that PS3DT performs well on ASVspoof2019 dataset compared to other approaches using spectrogram for synthetic speech detection. We also investigate generalization performance of PS3DT on In-the-Wild dataset. PS3DT generalizes well than several existing methods on detecting synthetic speech from an out-of-distribution dataset. We also evaluate robustness of PS3DT to detect telephone quality synthetic speech and synthetic speech shared on social platforms (compressed speech). PS3DT is robust to compression and can detect telephone quality synthetic speech better than several existing methods.


title: The impact of Facebook-Cambridge Analytica data scandal on the USA tech stock market: An event study based on clustering method

abstract: This study delves into the intra-industry effects following a firm-specific scandal, with a particular focus on the Facebook data leakage scandal and its associated events within the U.S. tech industry and two additional relevant groups. We employ various metrics including daily spread, volatility, volume-weighted return, and CAPM-beta for the pre-analysis clustering, and subsequently utilize CAR (Cumulative Abnormal Return) to evaluate the impact on firms grouped within these clusters. From a broader industry viewpoint, significant positive CAARs are observed across U.S. sample firms over the three days post-scandal announcement, indicating no adverse impact on the tech sector overall. Conversely, after Facebook's initial quarterly earnings report, it showed a notable negative effect despite reported positive performance. The clustering principle should aid in identifying directly related companies and thus reducing the influence of randomness. This was indeed achieved for the effect of the key event, namely "The Effect of Congressional Hearing on Certain Clusters across U.S. Tech Stock Market," which was identified as delayed and significantly negative. Therefore, we recommend applying the clustering method when conducting such or similar event studies.


title: Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models

abstract: We study how to apply large language models to write grounded and organized long-form articles from scratch, with comparable breadth and depth to Wikipedia pages. This underexplored problem poses new challenges at the pre-writing stage, including how to research the topic and prepare an outline prior to writing. We propose STORM, a writing system for the Synthesis of Topic Outlines through Retrieval and Multi-perspective Question Asking. STORM models the pre-writing stage by (1) discovering diverse perspectives in researching the given topic, (2) simulating conversations where writers carrying different perspectives pose questions to a topic expert grounded on trusted Internet sources, (3) curating the collected information to create an outline. For evaluation, we curate FreshWiki, a dataset of recent high-quality Wikipedia articles, and formulate outline assessments to evaluate the pre-writing stage. We further gather feedback from experienced Wikipedia editors. Compared to articles generated by an outline-driven retrieval-augmented baseline, more of STORM's articles are deemed to be organized (by a 25% absolute increase) and broad in coverage (by 10%). The expert feedback also helps identify new challenges for generating grounded long articles, such as source bias transfer and over-association of unrelated facts.


title: LLM-Assisted Content Conditional Debiasing for Fair Text Embedding

abstract: Mitigating biases in machine learning models has become an increasing concern in Natural Language Processing (NLP), particularly in developing fair text embeddings, which are crucial yet challenging for real-world applications like search engines. In response, this paper proposes a novel method for learning fair text embeddings. First, we define a novel content-conditional equal distance (CCED) fairness for text embeddings, ensuring content-conditional independence between sensitive attributes and text embeddings. Building on CCED, we introduce a content-conditional debiasing (CCD) loss to ensure that embeddings of texts with different sensitive attributes but identical content maintain the same distance from the embedding of their corresponding neutral text. Additionally, we tackle the issue of insufficient training data by using Large Language Models (LLMs) with instructions to fairly augment texts into different sensitive groups. Our extensive evaluations show that our approach effectively enhances fairness while maintaining the utility of embeddings. Furthermore, our augmented dataset, combined with the CCED metric, serves as an new benchmark for evaluating fairness.


title: Developing an Automated Detection, Tracking and Analysis Method for Solar Filaments Observed by CHASE via Machine Learning

abstract: Studies on the dynamics of solar filaments have significant implications for understanding their formation, evolution, and eruption, which are of great importance for space weather warning and forecasting. The H$\alpha$ Imaging Spectrograph (HIS) onboard the recently launched Chinese H$\alpha$ Solar Explorer (CHASE) can provide full-disk solar H$\alpha$ spectroscopic observations, which bring us an opportunity to systematically explore and analyze the plasma dynamics of filaments. The dramatically increased observation data require automate processing and analysis which are impossible if dealt with manually. In this paper, we utilize the U-Net model to identify filaments and implement the Channel and Spatial Reliability Tracking (CSRT) algorithm for automated filament tracking. In addition, we use the cloud model to invert the line-of-sight velocity of filaments and employ the graph theory algorithm to extract the filament spine, which can advance our understanding of the dynamics of filaments. The favorable test performance confirms the validity of our method, which will be implemented in the following statistical analyses of filament features and dynamics of CHASE/HIS observations.


title: Inclined junction in monolayer graphene: A gateway toward tailoring valley polarization of Dirac fermions

abstract: Generating discernible valley contrasts and segregating valley-indexed fermions in real space within graphene poses considerable challenges due to the isotropic transport within the continuum energy range for degenerate valleys. This study unveils an interesting finding: introducing valley contrast through anisotropic chiral transport in isotropic Dirac systems like graphene, achieved by implementing a tilted PN junction. The tilted junction shifts the angular spectrum to larger angles in accordance with the tilt angle. This modifies the pseudospin-conserved modes across the junction, resulting in valley-resolved chiral transport. This approach not only induces valley splitting within the real space but also preserves the remarkable mobility of fermions, offering distinct advantages over alternative strategies. The comprehensive analysis includes optimizing the experimental setup, scrutinizing factors such as the sequence of the doped region, and examining critical parameters like the tilt angle delta and transition width d across the junction. Surprisingly, an increased transition width enhances transmission, attributed to specular edge scattering. Importantly, the system remains resilient to Anderson short-range edge disorder. The broader implication lies in the transformative potential of inducing analogous anisotropic chiral transport behaviors in isotropic Dirac systems, resembling the characteristics of tilted Dirac-Weyl semimetals, by incorporating a tilted PNJ.


title: Induced subgraphs and tree decompositions XV. Even-hole-free graphs with bounded clique number have logarithmic treewidth

abstract: We prove that for every integer $t\geq 1$ there exists an integer $c_t\geq 1$ such that every $n$-vertex even-hole-free graph with no clique of size $t$ has treewidth at most $c_t\log{n}$. This resolves a conjecture of Sintiari and Trotignon, who also proved that the logarithmic bound is asymptotically best possible. It follows that several \textsf{NP}-hard problems such as \textsc{Stable Set}, \textsc{Vertex Cover}, \textsc{Dominating Set} and \textsc{Coloring} admit polynomial-time algorithms on this class of graphs. As a consequence, for every positive integer $r$, $r$-{\sc Coloring} can be solved in polynomial time on even-hole-free graphs without any assumptions on clique size. As part of the proof, we show that there is an integer $d$ such that every even-hole-free graph has a balanced separator which is contained in the (closed) neighborhood of at most $d$ vertices. This is of independent interest; for instance, it implies the existence of efficient approximation algorithms for certain \textsf{NP}-hard problems while restricted to the class of all even-hole-free graphs.


title: Moonwalk: Inverse-Forward Differentiation

abstract: Backpropagation, while effective for gradient computation, falls short in addressing memory consumption, limiting scalability. This work explores forward-mode gradient computation as an alternative in invertible networks, showing its potential to reduce the memory footprint without substantial drawbacks. We introduce a novel technique based on a vector-inverse-Jacobian product that accelerates the computation of forward gradients while retaining the advantages of memory reduction and preserving the fidelity of true gradients. Our method, Moonwalk, has a time complexity linear in the depth of the network, unlike the quadratic time complexity of na\"ive forward, and empirically reduces computation time by several orders of magnitude without allocating more memory. We further accelerate Moonwalk by combining it with reverse-mode differentiation to achieve time complexity comparable with backpropagation while maintaining a much smaller memory footprint. Finally, we showcase the robustness of our method across several architecture choices. Moonwalk is the first forward-based method to compute true gradients in invertible networks in computation time comparable to backpropagation and using significantly less memory.


title: Contrastive Learning of Shared Spatiotemporal EEG Representations Across Individuals for Naturalistic Neuroscience

abstract: Neural representations induced by naturalistic stimuli offer insights into how humans respond to peripheral stimuli in daily life. The key to understanding the general neural mechanisms underlying naturalistic stimuli processing involves aligning neural activities across individuals and extracting inter-subject shared neural representations. Targeting the Electroencephalogram (EEG) technique, known for its rich spatial and temporal information, this study presents a general framework for Contrastive Learning of Shared SpatioTemporal EEG Representations across individuals (CL-SSTER). Harnessing the representational capabilities of contrastive learning, CL-SSTER utilizes a neural network to maximize the similarity of EEG representations across individuals for identical stimuli, contrasting with those for varied stimuli. The network employed spatial and temporal convolutions to simultaneously learn the spatial and temporal patterns inherent in EEG. The versatility of CL-SSTER was demonstrated on three EEG datasets, including a synthetic dataset, a speech audio EEG dataset, and an emotional video EEG dataset. CL-SSTER attained the highest inter-subject correlation (ISC) values compared to the state-of-the-art ISC methods. The latent representations generated by CL-SSTER exhibited reliable spatiotemporal EEG patterns, which can be explained by specific aspects of the stimuli. CL-SSTER serves as an interpretable and scalable foundational framework for the identification of inter-subject shared neural representations in the realm of naturalistic neuroscience.


title: Ruijsenaars wavefunctions as modular group matrix coefficients

abstract: We give a description of the Halln\"as--Ruijsenaars eigenfunctions of the 2-particle hyperbolic Ruijsenaars system as matrix coefficients for the order 4 element $S\in SL(2,\mathbb{Z})$ acting on the Hilbert space of $GL(2)$ quantum Teichm\"uller theory on the punctured torus. The $GL(2)$ Macdonald polynomials are then obtained as special values of the analytic continuation of these matrix coefficients. The main tool used in the proof is the cluster structure on the moduli space of framed $GL(2)$-local systems on the punctured torus, and an $SL(2,\mathbb{Z})$-equivariant embedding of the $GL(2)$ spherical DAHA into the quantized coordinate ring of the corresponding cluster Poisson variety.


title: Swin3D++: Effective Multi-Source Pretraining for 3D Indoor Scene Understanding

abstract: Data diversity and abundance are essential for improving the performance and generalization of models in natural language processing and 2D vision. However, 3D vision domain suffers from the lack of 3D data, and simply combining multiple 3D datasets for pretraining a 3D backbone does not yield significant improvement, due to the domain discrepancies among different 3D datasets that impede effective feature learning. In this work, we identify the main sources of the domain discrepancies between 3D indoor scene datasets, and propose Swin3D++, an enhanced architecture based on Swin3D for efficient pretraining on multi-source 3D point clouds. Swin3D++ introduces domain-specific mechanisms to Swin3D's modules to address domain discrepancies and enhance the network capability on multi-source pretraining. Moreover, we devise a simple source-augmentation strategy to increase the pretraining data scale and facilitate supervised pretraining. We validate the effectiveness of our design, and demonstrate that Swin3D++ surpasses the state-of-the-art 3D pretraining methods on typical indoor scene understanding tasks. Our code and models will be released at https://github.com/microsoft/Swin3D


title: An asymptotic property on a reciprocity law for the Bettin--Conrey cotangent sum

abstract: In 2013 Bettin and Conrey have introduced a cotangent sum $c \colon \mathbb{Q}_{>0}\to \mathbb{R}$, which can be regarded as a variant of the Dedekind sum. They have discovered that the cotangent sum satisfies a kind of reciprocity laws. Roughly speaking, the reciprocity law for $c(x)$ means that there is a relation between $c(x)$ and $c(1/x)$ modulo holomorphic functions. Furthermore they have investigated Taylor coefficients $g_n$ of the implicit holomorphic function, which appears in the reciprocity law for $c(x)$, at $x=1$. As a result, they have obtained an asymptotic formula for $g_n$ as $n\to\infty$. In this paper we improve it to an asymptotic series expansion. This resolves a conjecture by Zagier. A new ingredient of this paper is to use the confluent hypergeometric function of the second kind.


title: The diagonal derivative of a skew Schur polynomial

abstract: We prove a formula for the image of a skew Schur polynomial $s_{\lambda/\mu}\left( x_{1}, x_{2}, \ldots, x_{N}\right) $ under the differential operator $\nabla:= \dfrac{\partial}{\partial x_{1}} +\dfrac{\partial}{\partial x_{2}}+\cdots+\dfrac{\partial}{\partial x_{N}}$. This generalizes a formula of Weigandt for $\nabla\left( s_{\lambda}\right) $.


title: Velocity Dispersions of Quiescent Galaxies in IllustirisTNG

abstract: We examine the central stellar velocity dispersion of subhalos based on IllustrisTNG cosmological hydrodynamic simulations. The central velocity dispersion is a fundamental observable that links galaxies with their dark matter subhalos. We carefully explore simulated stellar velocity dispersions derived with different definitions to assess possible systematics. We explore the impact of variation in the identification of member stellar particles, the viewing axes, the velocity dispersion computation technique, and simulation resolution. None of these issues impact the velocity dispersion significantly; any systematic uncertainties are smaller than the random error. We examine the stellar mass-velocity dispersion relation as an observational test of the simulations. At fixed stellar mass, the observed velocity dispersions significantly exceed the simulation results. This discrepancy is an interesting benchmark for the IllustrisTNG simulations because the simulations are not explicitly tuned to match this relation. We demonstrate that the stellar velocity dispersion provides measures of the dark matter velocity dispersion and the dark matter subhalo mass.


title: Large Array Antenna Spectrum Sensing in Cognitive Radio Networks

abstract: We investigate the problem of spectrum sensing in cognitive radios (CRs) when the receivers are equipped with a large array of antennas. We propose and derive three detectors based on the concept of linear spectral statistics (LSS) in the field of random matrix theory (RMT). These detectors correspond to the generalized likelihood ratio (GLR), Frobenius norm, and Rao tests employed in conventional multiple antenna spectrum sensing (MASS). Subsequently, we compute the Gaussian distribution of the proposed detectors under the noise-only hypothesis, leveraging the central limit theorem (CLT) applied to high-dimensional random matrices. We evaluate the performance of the proposed detectors and analyze the impact of the number of antennas and samples on their efficacy. Furthermore, we assess the accuracy of the theoretical results by comparing them with simulation outcomes. The simulation results provide evidence that the proposed detectors exhibit efficient performance in wireless networks featuring large array antennas. These detectors find practical applications in diverse domains, including massive MIMO wireless communications, radar systems, and astronomical applications.


title: Estimating Unknown Population Sizes Using the Hypergeometric Distribution

abstract: The multivariate hypergeometric distribution describes sampling without replacement from a discrete population of elements divided into multiple categories. Addressing a gap in the literature, we tackle the challenge of estimating discrete distributions when both the total population size and the sizes of its constituent categories are unknown. Here, we propose a novel solution using the hypergeometric likelihood to solve this estimation challenge, even in the presence of severe under-sampling. We develop our approach to account for a data generating process where the ground-truth is a mixture of distributions conditional on a continuous latent variable, such as with collaborative filtering, using the variational autoencoder framework. Empirical data simulation demonstrates that our method outperforms other likelihood functions used to model count data, both in terms of accuracy of population size estimate and in its ability to learn an informative latent space. We demonstrate our method's versatility through applications in NLP, by inferring and estimating the complexity of latent vocabularies in text excerpts, and in biology, by accurately recovering the true number of gene transcripts from sparse single-cell genomics data.


title: Towards singular optimality in the presence of local initial knowledge

abstract: The Knowledge Till rho CONGEST model is a variant of the classical CONGEST model of distributed computing in which each vertex v has initial knowledge of the radius-rho ball centered at v. The most commonly studied variants of the CONGEST model are KT0 CONGEST in which nodes initially know nothing about their neighbors and KT1 CONGEST in which nodes initially know the IDs of all their neighbors. It has been shown that having access to neighbors' IDs (as in the KT1 CONGEST model) can substantially reduce the message complexity of algorithms for fundamental problems such as BROADCAST and MST. For example, King, Kutten, and Thorup (PODC 2015) show how to construct an MST using just Otilde(n) messages in the KT1 CONGEST model, whereas there is an Omega(m) message lower bound for MST in the KT0 CONGEST model. Building on this result, Gmyr and Pandurangen (DISC 2018) present a family of distributed randomized algorithms for various global problems that exhibit a trade-off between message and round complexity. These algorithms are based on constructing a sparse, spanning subgraph called a danner. Specifically, given a graph G and any delta in [0,1], their algorithm constructs (with high probability) a danner that has diameter Otilde(D + n^{1-delta}) and Otilde(min{m,n^{1+delta}}) edges in Otilde(n^{1-delta}) rounds while using Otilde(min{m,n^{1+\delta}}) messages, where n, m, and D are the number of nodes, edges, and the diameter of G, respectively. In the main result of this paper, we show that if we assume the KT2 CONGEST model, it is possible to substantially improve the time-message trade-off in constructing a danner. Specifically, we show in the KT2 CONGEST model, how to construct a danner that has diameter Otilde(D + n^{1-2delta}) and Otilde(min{m,n^{1+delta}}) edges in Otilde(n^{1-2delta}) rounds while using Otilde(min{m,n^{1+\delta}}) messages for any delta in [0,1/2].


title: Michael's selection theorem in general d-minimal structures

abstract: Thamrongthanyalak demonstrated a definable version of Michael's selection theorem in d-minimal expansions of the real field. We generalize this result to the case in which the structures are d-minimal expansions of ordered fields $\mathcal F=(F,<,+,\cdot,0,1,\ldots)$. We also show that we can choose a definable continuous selection $f$ of a lower semi-continuous map $T:E \rightrightarrows F$ so that $f(x)$ is contained in the interior of $T(x)$ when the interior is not empty.


title: Interacting light thermal-relic dark matter: self-consistent cosmological bounds

abstract: We analyze cosmic microwave background (CMB) data to constrain the mass and interaction strengths of thermally-produced dark matter (DM) in a self-consistent manner, simultaneously taking into account the cosmological effects of its mass and interactions. The presence of a light thermal-relic particle contributes non-negligibly to the radiation density during Big Bang Nucleosynthesis (BBN), altering the light-element yields, as well as the the effective number of relativistic particle species. On the other hand, DM interactions with the Standard Model can affect distribution of matter in later universe. Both mass and interactions alter CMB anisotropy on sub-degree scales. To understand and quantify the interplay of these effects, we consider elastic DM-baryon scattering with a momentum-transfer cross section that scales as a power law of the relative velocity between the scattering particles. In the range of thermal-relic DM masses relevant for BBN ($\lesssim$ 20 MeV), we find that the reconstruction of the DM mass and the scattering cross section from the CMB data features strong degeneracies; modeling the two effects simultaneously increases the sensitivity of the CMB measurements to both fundamental properties of DM. Additionally, we study the effects of late-time residual annihilation of a light thermal relic and provide improved CMB constraints on the DM mass and annihilation cross section. To examine degeneracy between DM mass, cross section for elastic scattering with baryons, and annihilation cross section, we consider a specific case of DM with an electric and magnetic dipole moments. We present new, self-consistent cosmological bounds for this model and discuss implications for future searches.


title: Framing in the Presence of Supporting Data: A Case Study in U.S. Economic News

abstract: The mainstream media has much leeway in what it chooses to cover and how it covers it. These choices have real-world consequences on what people know and their subsequent behaviors. However, the lack of objective measures to evaluate editorial choices makes research in this area particularly difficult. In this paper, we argue that there are newsworthy topics where objective measures exist in the form of supporting data and propose a computational framework to analyze editorial choices in this setup. We focus on the economy because the reporting of economic indicators presents us with a relatively easy way to determine both the selection and framing of various publications. Their values provide a ground truth of how the economy is doing relative to how the publications choose to cover it. To do this, we define frame prediction as a set of interdependent tasks. At the article level, we learn to identify the reported stance towards the general state of the economy. Then, for every numerical quantity reported in the article, we learn to identify whether it corresponds to an economic indicator and whether it is being reported in a positive or negative way. To perform our analysis, we track six American publishers and each article that appeared in the top 10 slots of their landing page between 2015 and 2023.


title: SICRN: Advancing Speech Enhancement through State Space Model and Inplace Convolution Techniques

abstract: Speech enhancement aims to improve speech quality and intelligibility, especially in noisy environments where background noise degrades speech signals. Currently, deep learning methods achieve great success in speech enhancement, e.g. the representative convolutional recurrent neural network (CRN) and its variants. However, CRN typically employs consecutive downsampling and upsampling convolution for frequency modeling, which destroys the inherent structure of the signal over frequency. Additionally, convolutional layers lacks of temporal modelling abilities. To address these issues, we propose an innovative module combing a State space model and Inplace Convolution (SIC), and to replace the conventional convolution in CRN, called SICRN. Specifically, a dual-path multidimensional State space model captures the global frequencies dependency and long-term temporal dependencies. Meanwhile, the 2D-inplace convolution is used to capture the local structure, which abandons the downsampling and upsampling. Systematic evaluations on the public INTERSPEECH 2020 DNS challenge dataset demonstrate SICRN's efficacy. Compared to strong baselines, SICRN achieves performance close to state-of-the-art while having advantages in model parameters, computations, and algorithmic delay. The proposed SICRN shows great promise for improved speech enhancement.


title: Broadband noise and quasi-periodic oscillation characteristics of the X-ray pulsar RX J0440.9+4431

abstract: We present a comprehensive timing analysis on the Be/X-ray binary pulsar RX J0440.9+4431 using observations from \textit{NICER} and \textit{Insight}-HXMT during the 2022--2023 outburst. The power density spectrum (PDS) of RX J0440.9+4431 exhibits typical aperiodic variability in X-ray flux across a wide frequency range. During a super-critical accretion state, we detect quasi-periodic oscillations (QPOs) at 0.2--0.5\,Hz in the light curves of five pulses for RX J0440.9+4431. The observed QPOs manifest during flares, while the flares appear at the peaks of the pulse profiles on a timescale of seconds and are primarily caused by an increase in hard photons. These flares can be explained by increased material ingestion in the accretion column at a fixed phase, primarily generating hard photons. Alternatively, an increase in accretion rate, independent of phase, may result in highly beamed hard photons within the accretion column, causing the flares. We argue the origin of QPOs to instabilities within the accretion flow. Additionally, we find that the break frequencies in the noise power spectra align well with $\propto L_{\mathrm{x}}^{3 / 7}$ across three orders of magnitude in the luminosity, which points to a relatively strong magnetic field in RX J0440.9+4431, estimated to be \textasciitilde$10^{13}$\,G.


title: Quaternion recurrent neural network with real-time recurrent learning and maximum correntropy criterion

abstract: We develop a robust quaternion recurrent neural network (QRNN) for real-time processing of 3D and 4D data with outliers. This is achieved by combining the real-time recurrent learning (RTRL) algorithm and the maximum correntropy criterion (MCC) as a loss function. While both the mean square error and maximum correntropy criterion are viable cost functions, it is shown that the non-quadratic maximum correntropy loss function is less sensitive to outliers, making it suitable for applications with multidimensional noisy or uncertain data. Both algorithms are derived based on the novel generalised HR (GHR) calculus, which allows for the differentiation of real functions of quaternion variables and offers the product and chain rules, thus enabling elegant and compact derivations. Simulation results in the context of motion prediction of chest internal markers for lung cancer radiotherapy, which includes regular and irregular breathing sequences, support the analysis.


title: COPR: Continual Human Preference Learning via Optimal Policy Regularization

abstract: Reinforcement Learning from Human Feedback (RLHF) is commonly utilized to improve the alignment of Large Language Models (LLMs) with human preferences. Given the evolving nature of human preferences, continual alignment becomes more crucial and practical in comparison to traditional static alignment. Nevertheless, making RLHF compatible with Continual Learning (CL) is challenging due to its complex process. Meanwhile, directly learning new human preferences may lead to Catastrophic Forgetting (CF) of historical preferences, resulting in helpless or harmful outputs. To overcome these challenges, we propose the Continual Optimal Policy Regularization (COPR) method, which draws inspiration from the optimal policy theory. COPR utilizes a sampling distribution as a demonstration and regularization constraints for CL. It adopts the Lagrangian Duality (LD) method to dynamically regularize the current policy based on the historically optimal policy, which prevents CF and avoids over-emphasizing unbalanced objectives. We also provide formal proof for the learnability of COPR. The experimental results show that COPR outperforms strong CL baselines on our proposed benchmark, in terms of reward-based, GPT-4 evaluations and human assessment. Furthermore, we validate the robustness of COPR under various CL settings, including different backbones, replay memory sizes, and learning orders.


title: Sample-Efficient Linear Regression with Self-Selection Bias

abstract: We consider the problem of linear regression with self-selection bias in the unknown-index setting, as introduced in recent work by Cherapanamjeri, Daskalakis, Ilyas, and Zampetakis [STOC 2023]. In this model, one observes $m$ i.i.d. samples $(\mathbf{x}_{\ell},z_{\ell})_{\ell=1}^m$ where $z_{\ell}=\max_{i\in [k]}\{\mathbf{x}_{\ell}^T\mathbf{w}_i+\eta_{i,\ell}\}$, but the maximizing index $i_{\ell}$ is unobserved. Here, the $\mathbf{x}_{\ell}$ are assumed to be $\mathcal{N}(0,I_n)$ and the noise distribution $\mathbf{\eta}_{\ell}\sim \mathcal{D}$ is centered and independent of $\mathbf{x}_{\ell}$. We provide a novel and near optimally sample-efficient (in terms of $k$) algorithm to recover $\mathbf{w}_1,\ldots,\mathbf{w}_k\in \mathbb{R}^n$ up to additive $\ell_2$-error $\varepsilon$ with polynomial sample complexity $\tilde{O}(n)\cdot \mathsf{poly}(k,1/\varepsilon)$ and significantly improved time complexity $\mathsf{poly}(n,k,1/\varepsilon)+O(\log(k)/\varepsilon)^{O(k)}$. When $k=O(1)$, our algorithm runs in $\mathsf{poly}(n,1/\varepsilon)$ time, generalizing the polynomial guarantee of an explicit moment matching algorithm of Cherapanamjeri, et al. for $k=2$ and when it is known that $\mathcal{D}=\mathcal{N}(0,I_k)$. Our algorithm succeeds under significantly relaxed noise assumptions, and therefore also succeeds in the related setting of max-linear regression where the added noise is taken outside the maximum. For this problem, our algorithm is efficient in a much larger range of $k$ than the state-of-the-art due to Ghosh, Pananjady, Guntuboyina, and Ramchandran [IEEE Trans. Inf. Theory 2022] for not too small $\varepsilon$, and leads to improved algorithms for any $\varepsilon$ by providing a warm start for existing local convergence methods.


title: MerRec: A Large-scale Multipurpose Mercari Dataset for Consumer-to-Consumer Recommendation Systems

abstract: In the evolving e-commerce field, recommendation systems crucially shape user experience and engagement. The rise of Consumer-to-Consumer (C2C) recommendation systems, noted for their flexibility and ease of access for customer vendors, marks a significant trend. However, the academic focus remains largely on Business-to-Consumer (B2C) models, leaving a gap filled by the limited C2C recommendation datasets that lack in item attributes, user diversity, and scale. The intricacy of C2C recommendation systems is further accentuated by the dual roles users assume as both sellers and buyers, introducing a spectrum of less uniform and varied inputs. Addressing this, we introduce MerRec, the first large-scale dataset specifically for C2C recommendations, sourced from the Mercari e-commerce platform, covering millions of users and products over 6 months in 2023. MerRec not only includes standard features such as user_id, item_id, and session_id, but also unique elements like timestamped action types, product taxonomy, and textual product attributes, offering a comprehensive dataset for research. This dataset, extensively evaluated across six recommendation tasks, establishes a new benchmark for the development of advanced recommendation algorithms in real-world scenarios, bridging the gap between academia and industry and propelling the study of C2C recommendations.


title: On Gibbs Equilibrium and Hillert Nonequilibrium Thermodynamics

abstract: During his time at Royal Institute of Technology (Kungliga Tekniska Hogskolan, KTH) in Sweden, the present author learned nonequilibrium thermodynamics from Mats Hillert. The key concept is the separation of internal and external variables of a system. In equilibrium thermodynamics derived by Gibbs, the internal variables are not independent and can be fully evaluated from given external variables. While irreversible thermodynamics led by Onsager focuses on internal variables though often mixed with external variables. Hillert integrated them together by first emphasizing their differences and then examining their connections. His philosophy was reflected by the title of his book "Phase Equilibria, Phase Diagrams and Phase Transformations" that puts equilibrium, nonequilibrium, and internal processes on equal footing. In the present paper honoring Hillert, the present author reflects his experiences with Hillert in last 35 years and expresses his gratitude for all the wisdom and support from him in terms of "Hillert nonequilibrium thermodynamics" and discusses some recent topics that the present author has been working on.


title: The quark flavor-violating ALPs in light of B mesons and hadron colliders

abstract: The axion-like particle (ALP) may induce flavor-changing neutral currents (FCNCs) when their Peccei-Quinn charges are not generation universal. The search for flavor-violating ALP couplings with a bottom quark so far focused on FCNC processes of $B$ mesons at low energies. The recent measurements of $B\to K +X$ rare decays place stringent bounds on the quark flavor violations of a light ALP in different decay modes. In this work we propose a novel direct search for bottom flavor-violating interaction of a heavy ALP at the LHC and its upgrades, namely QCD production of an ALP associated with one $b$ jet and one light jet $p~p\to b~j~a$. We consider the decay of the ALP to photons, muons and invisible ALP decays. The Boosted Decision Tree (BDT) algorithm is used to analyze the events and we train the BDT classifier by feeding in the kinematic observables of signal and backgrounds. Finally, we show the complementarity between the search prospects of hadron colliders and the low-energy $B$ meson constraints from $B$ meson mixing and $B$ meson decays to a light ALP.


title: Stand-Up Indulgent Gathering on Rings

abstract: We consider a collection of $k \geq 2$ robots that evolve in a ring-shaped network without common orientation, and address a variant of the crash-tolerant gathering problem called the \emph{Stand-Up Indulgent Gathering} (SUIG): given a collection of robots, if no robot crashes, robots have to meet at the same arbitrary location, not known beforehand, in finite time; if one robot or more robots crash on the same location, the remaining correct robots gather at the location of the crashed robots. We aim at characterizing the solvability of the SUIG problem without multiplicity detection capability.


title: The sum of the reciprocals of the prime divisors of an odd perfect or odd primitive non-deficient number

abstract: Write $T(n)$ as the sum of the reciprocals of the primes which divide $n$. Write $H(n) = \prod_{p|n}p/(p-1)$ where the product is over the prime divisors of $n$. We prove new bounds for $T(n)$ and $H(n)$ in terms of the smallest prime factor of $n$, under the assumption that $n$ is an odd perfect number. Some of the results also apply under the weaker assumption that $n$ is odd and primitive non-deficient.


title: Lagrangian partition functions subject to a fixed spatial volume constraint in the Lovelock theory

abstract: We evaluate the quantum gravity partition function that counts the dimension of the Hilbert space of a simply connected spatial region of fixed proper volume in the context of Lovelock gravity, generalizing the results for Einstein gravity [1]. We find that there exists sphere saddle metrics for a partition function at a fixed spatial volume in Lovelock theory. Those stationary points take exactly the same forms as in Einstein gravity. The logarithm of Z corresponding to a zero effective cosmological constant indicates the Bekenstein-Hawking entropy of the boundary area and the one corresponding to a positive effective cosmological constant points to the Wald entropy of the boundary area. We also observe the existence of zeroth order phase transitions between different vacua, a phenomenon distinct from Einstein gravity.


title: Automated Design and Optimization of Distributed Filtering Circuits via Reinforcement Learning

abstract: Designing distributed filtering circuits (DFCs) is complex and time-consuming, with the circuit performance relying heavily on the expertise and experience of electronics engineers. However, manual design methods tend to have exceedingly low-efficiency. This study proposes a novel end-to-end automated method for fabricating circuits to improve the design of DFCs. The proposed method harnesses reinforcement learning (RL) algorithms, eliminating the dependence on the design experience of engineers. Thus, it significantly reduces the subjectivity and constraints associated with circuit design. The experimental findings demonstrate clear improvements in both design efficiency and quality when comparing the proposed method with traditional engineer-driven methods. In particular, the proposed method achieves superior performance when designing complex or rapidly evolving DFCs. Furthermore, compared to existing circuit automation design techniques, the proposed method demonstrates superior design efficiency, highlighting the substantial potential of RL in circuit design automation.


title: The Generalized Gaussian Minkowski Problem

abstract: This article studies the $L_p$ Minkowski problem in generalized Gaussian probability space, where this kind of probability space was first proposed in information theory by the remarkable works [47] by Lutwak-Yang-Zhang and [42] by Lutwak-Lv-Yang-Zhang. The main focus of this article is on the existence of this problem. While the variational method is used to consider the necessary and sufficient conditions for the existence of normalized Minkowski problem when $p\in \mathbb{R}\setminus\{0\}$, we mainly focus on the existence of generalized Gaussian Minkowski problem without normalization required in smooth category for $p\geq1$. In addition, there are the Brunn-Minkowski inequalities about the generalized Gaussian volume.


title: Optical properties of two dimensional Dirac Weyl materials with a flatband

abstract: The emergence of a flat band in Dirac-Weyl materials offers new possibilities for electronic transitions, leading to stronger interaction with light. As a result, the optical conductivity can be significantly enhanced in these flat-band materials as compared with graphene, making them potentially better candidates for optical sensing and modulation. Recently, a comprehensive theory for the optical conductivity of a spectrum of flat-band Dirac-Weyl materials has been developed, with explicit formulas for both the real and imaginary parts of the conductivity derived through two independent approaches. This Perspective offers a review of the development. An understanding of the optical properties of the flat-band Dirac-Weyl materials paves the way for optical device applications in the terahertz-frequency domain.


title: Velocity recostruction with graph neural networks

abstract: In this work, we seek to improve the velocity reconstruction of clusters by using Graph Neural Networks -- a type of deep neural network designed to analyze sparse, unstructured data. In comparison to the Convolutional Neural Network (CNN) which is built for structured data such as regular grids, GNN is particularly suitable for analyzing galaxy catalogs. In our GNNs, galaxies as represented as nodes that are connected with edges. The galaxy positions and properties -- stellar mass, star formation rate, and total number of galaxies within 100~\mpc -- are combined to predict the line-of-sight velocity of the clusters. To train our networks, we use mock SDSS galaxies and clusters constructed from the Magneticum hydrodynamic simulations. Our GNNs reach a precision in reconstructed line-of-sight velocity of $\Delta v$=163 km/s, outperforming by $\approx$10\% the perturbation theory~($\Delta v$=181 km/s) or the CNN~($\Delta v$=179 km/s). The stellar mass provides additional information, improving the precision by $\approx$6\% beyond the position-only GNN, while other properties add little information. Our GNNs remain capable of reconstructing the velocity field when redshift-space distortion is included, with $\Delta v$=210 km/s which is again 10\% better than CNN with RSD. Finally, we find that even with an impressive, nearly 70\% increase in galaxy number density from SDSS to DESI, our GNNs only show an underwhelming 2\% improvement, in line with previous works using other methods. Our work demonstrates that, while the efficiency in velocity reconstruction may have plateaued already at SDSS number density, further improvements are still hopeful with new reconstruction models such as the GNNs studied here.


title: A Principle of Maximum Entropy for the Navier-Stokes Equations

abstract: A principle of maximum entropy is proposed in the context of viscous incompressible flow in Eulerian coordinates. The relative entropy functional, defined over the space of $L^2$ divergence-free velocity fields, is maximized relative to alternate measures supported over the energy--enstrophy surface. Since thermodynamic equilibrium distributions are characterized by maximum entropy, connections are drawn with stationary statistical solutions of the incompressible Navier-Stokes equations. Special emphasis is on the correspondence with the final statistics described by Kolmogorov's theory of fully developed turbulence.

categories: physics.flu-dyn math-ph math.AP math.MP physics.class-ph


title: A Self-supervised Pressure Map human keypoint Detection Approch: Optimizing Generalization and Computational Efficiency Across Datasets

abstract: In environments where RGB images are inadequate, pressure maps is a viable alternative, garnering scholarly attention. This study introduces a novel self-supervised pressure map keypoint detection (SPMKD) method, addressing the current gap in specialized designs for human keypoint extraction from pressure maps. Central to our contribution is the Encoder-Fuser-Decoder (EFD) model, which is a robust framework that integrates a lightweight encoder for precise human keypoint detection, a fuser for efficient gradient propagation, and a decoder that transforms human keypoints into reconstructed pressure maps. This structure is further enhanced by the Classification-to-Regression Weight Transfer (CRWT) method, which fine-tunes accuracy through initial classification task training. This innovation not only enhances human keypoint generalization without manual annotations but also showcases remarkable efficiency and generalization, evidenced by a reduction to only $5.96\%$ in FLOPs and $1.11\%$ in parameter count compared to the baseline methods.


title: Non-equilibrium theory of the linear viscoelasticity of glass and gel forming liquids

abstract: We propose a first-principles theoretical approach for the description of the aging of the linear viscoelastic properties of a colloidal liquid after a sudden quench into a dynamically arrested (glass or gel) state. Specifically, we couple a general expression for the time-evolving shear-stress relaxation function $\eta(\tau;t)$ (whose $\tau$-integral is the instantaneous viscosity $\eta(t)$), written in terms of the non-equilibrium structure factor $S(k;t)$ and intermediate scattering function $F(k,\tau;t)$, with the equations that determine $S(k;t)$ and $F(k,\tau;t)$, provided by the non-equilibrium self-consistent generalized Langevin equation (NE-SCGLE) theory. In this manner, we obtain a closed theoretical scheme that directly connects inter-particle forces with experimentally accessible rheological properties of non-equilibrium amorphous states of matter. The predictive capability of the resulting theoretical formalism is illustrated here with its concrete application to the Weeks-Chandler-Andersen (WCA) model of a soft-sphere fluid.


title: Irregular Bloch Zener oscillations in two-dimensional flat-band Dirac materials

abstract: When a static electrical field is applied to a two-dimensional (2D) Dirac material, Landau-Zener transition (LZT) and Bloch-Zener oscillations can occur. Employing alpha-T3 lattices as a paradigm for a broad class of 2D Dirac materials, we uncover two phenomena. First, due to the arbitrarily small energy gaps near a Dirac point that make it more likely for LZTs to occur than in other regions of the Brillouin zone, the distribution of differential LZT probability in the momentum space can form a complicated morphological pattern. Second, a change in the LZT morphology as induced by a mutual switching of the two distinct Dirac points can lead to irregular Bloch-Zener oscillations characterized by a non-smooth behavior in the time evolution of the electrical current density associated with the oscillation. These phenomena are due to mixed interference of quantum states in multiple bands modulated by the geometric and dynamic phases. We demonstrate that the adiabatic-impulse model describing Landau-Zener-Stuckelberg interferometry can be exploited to calculate the phases, due to the equivalence between the alpha-T3 lattice subject to a constant electrical field and strongly periodically driven two- or three-level systems. The degree of irregularity of Bloch-Zener oscillations can be harnessed by selecting the morphology pattern, which is potentially experimentally realizable.


title: MENTOR: Guiding Hierarchical Reinforcement Learning with Human Feedback and Dynamic Distance Constraint

abstract: Hierarchical reinforcement learning (HRL) provides a promising solution for complex tasks with sparse rewards of intelligent agents, which uses a hierarchical framework that divides tasks into subgoals and completes them sequentially. However, current methods struggle to find suitable subgoals for ensuring a stable learning process. Without additional guidance, it is impractical to rely solely on exploration or heuristics methods to determine subgoals in a large goal space. To address the issue, We propose a general hierarchical reinforcement learning framework incorporating human feedback and dynamic distance constraints (MENTOR). MENTOR acts as a "mentor", incorporating human feedback into high-level policy learning, to find better subgoals. As for low-level policy, MENTOR designs a dual policy for exploration-exploitation decoupling respectively to stabilize the training. Furthermore, although humans can simply break down tasks into subgoals to guide the right learning direction, subgoals that are too difficult or too easy can still hinder downstream learning efficiency. We propose the Dynamic Distance Constraint (DDC) mechanism dynamically adjusting the space of optional subgoals. Thus MENTOR can generate subgoals matching the low-level policy learning process from easy to hard. Extensive experiments demonstrate that MENTOR uses a small amount of human feedback to achieve significant improvement in complex tasks with sparse rewards.


title: Enhancing Robotic Manipulation with AI Feedback from Multimodal Large Language Models

abstract: Recently, there has been considerable attention towards leveraging large language models (LLMs) to enhance decision-making processes. However, aligning the natural language text instructions generated by LLMs with the vectorized operations required for execution presents a significant challenge, often necessitating task-specific details. To circumvent the need for such task-specific granularity, inspired by preference-based policy learning approaches, we investigate the utilization of multimodal LLMs to provide automated preference feedback solely from image inputs to guide decision-making. In this study, we train a multimodal LLM, termed CriticGPT, capable of understanding trajectory videos in robot manipulation tasks, serving as a critic to offer analysis and preference feedback. Subsequently, we validate the effectiveness of preference labels generated by CriticGPT from a reward modeling perspective. Experimental evaluation of the algorithm's preference accuracy demonstrates its effective generalization ability to new tasks. Furthermore, performance on Meta-World tasks reveals that CriticGPT's reward model efficiently guides policy learning, surpassing rewards based on state-of-the-art pre-trained representation models.


title: Reconstruction-Based Anomaly Localization via Knowledge-Informed Self-Training

abstract: Anomaly localization, which involves localizing anomalous regions within images, is a significant industrial task. Reconstruction-based methods are widely adopted for anomaly localization because of their low complexity and high interpretability. Most existing reconstruction-based methods only use normal samples to construct model. If anomalous samples are appropriately utilized in the process of anomaly localization, the localization performance can be improved. However, usually only weakly labeled anomalous samples are available, which limits the improvement. In many cases, we can obtain some knowledge of anomalies summarized by domain experts. Taking advantage of such knowledge can help us better utilize the anomalous samples and thus further improve the localization performance. In this paper, we propose a novel reconstruction-based method named knowledge-informed self-training (KIST) which integrates knowledge into reconstruction model through self-training. Specifically, KIST utilizes weakly labeled anomalous samples in addition to the normal ones and exploits knowledge to yield pixel-level pseudo-labels of the anomalous samples. Based on the pseudo labels, a novel loss which promotes the reconstruction of normal pixels while suppressing the reconstruction of anomalous pixels is used. We conduct experiments on different datasets and demonstrate the advantages of KIST over the existing reconstruction-based methods.


title: Spectrum of the Dirac operator on Compact Riemannian Manifolds

abstract: In this paper, we consider the eigenvalue problem of Dirac operator on a compact Riemannian manifold isometrically immersed into Euclidean space and derive some extrinsic estimates for the sum of arbitrary consecutive $n$ eigenvalues of the square of the Dirac operator acting on some Dirac invariant subbundles. As some applications, we deduce some eigenvalue inequalities on the compact submanifolds immersed into Euclidean space, unit sphere or projective spaces and further get some bounds of general Reilly type. In addition, we also establish some universal bounds under certain curvature condition and on the meanwhile provide an alternative proof for Anghel's result. In particular, utilizing Atiyah-Singer index theorem, we drive an upper bound estimate for the sum of the first $n$ nontrivial eigenvalues of Atiyah-Singer Laplacian acting on the spin manifolds without dimensional assumption.


title: Spin-dependent edge states in two-dimensional Dirac materials with a flat band

abstract: The phenomenon of spin-dependent quantum scattering in two-dimensional (2D) pseudospin-1/2 Dirac materials leading to a relativistic quantum chimera was recently uncovered. We investigate spin-dependent Dirac electron optics in 2D pseudospin-1 Dirac materials, where the energy-band structure consists of a pair of Dirac cones and a flat band. In particular, with a suitable combination of external electric fields and a magnetic exchange field, electrons with a specific spin orientation (e.g., spin-down) can be trapped in a class of long-lived edge modes, generating resonant scattering. The spin-dependent edge states are a unique feature of flat-band Dirac materials and have no classical correspondence. However, electrons with the opposite spin (i.e., spin up) undergo conventional quantum scattering with a classical correspondence, which can be understood in the framework of Dirac electron optics. A consequence is that the spin-down electrons produce a large scattering probability with broad scattering angle distribution in both near- and far-field regions, while the spin-up electrons display the opposite behavior. Such characteristically different behaviors of the electrons with opposite spins lead to spin polarization that can be as high as nearly 100%.


title: On the Cohomology of Restricted Heisenberg Lie Algebras

abstract: We show that the Heisenberg Lie algebras over an algebraically closed field $\mathbb{F}$ of characteristic $p>0$ admit a family of restricted Lie algebras. We use the ordinary 1- and 2-cohomology spaces with trivial coefficients to compute the restricted 1- and 2-cohomology spaces of these restricted Heisenberg Lie algebras. We describe the restricted 1-dimensional central extensions, including explicit formulas for the Lie brackets and $\cdot^{[p]}$-operators.


title: Anomalies in Light Scattering: A Circuit Model Approach

abstract: In experimental physics, it is essential to understand electromagnetic (EM) wave scattering across EM spectrum, from radio waves to X-rays, and is pivotal in driving photonics innovations. Recent advancements have uncovered phenomena like bound states in the continuum (BICs) and parity-time (PT) symmetric systems, which are closely associated with the characteristics of the scattering matrix and are governed by passivity and causality. The emergence of complex frequency excitations has transcended the constraints imposed by passivity and causality in a system, revealing effects such as virtual critical coupling and virtual gain. However, applying the concepts of complex frequency excitation in more complicated systems remains challenging. In this work, we demonstrate the extension of the lumped element model of circuit theory to the analysis of anomalies in light scattering in the complex frequency domain. We demonstrate that the circuit model approach can facilitate design and analysis of effects such as virtual perfect absorption, BICs, real and virtual critical coupling, exceptional points, and anisotropic transmission resonances (ATRs). These findings broaden comprehension of EM wave phenomena and pave the way for significant advancements in photonics, offering new methods for designing and optimizing optical devices and systems with broad-ranging applications.


title: Make Interaction Situated: Designing User Acceptable Interaction for Situated Visualization in Public Environments

abstract: Situated visualization blends data into the real world to fulfill individuals' contextual information needs. However, interacting with situated visualization in public environments faces challenges posed by user acceptance and contextual constraints. To explore appropriate interaction design, we first conduct a formative study to identify user needs for data and interaction. Informed by the findings, we summarize appropriate interaction modalities with eye-based, hand-based and spatially-aware object interaction for situated visualization in public environments. Then, through an iterative design process with six users, we explore and implement interactive techniques for activating and analyzing with situated visualization. To assess the effectiveness and acceptance of these interactions, we integrate them into an AR prototype and conduct a within-subjects study in public scenarios using conventional hand-only interactions as the baseline. The results show that participants preferred our prototype over the baseline, attributing their preference to the interactions being more acceptable, flexible, and practical in public.


title: Multimodal Healthcare AI: Identifying and Designing Clinically Relevant Vision-Language Applications for Radiology

abstract: Recent advances in AI combine large language models (LLMs) with vision encoders that bring forward unprecedented technical capabilities to leverage for a wide range of healthcare applications. Focusing on the domain of radiology, vision-language models (VLMs) achieve good performance results for tasks such as generating radiology findings based on a patient's medical image, or answering visual questions (e.g., 'Where are the nodules in this chest X-ray?'). However, the clinical utility of potential applications of these capabilities is currently underexplored. We engaged in an iterative, multidisciplinary design process to envision clinically relevant VLM interactions, and co-designed four VLM use concepts: Draft Report Generation, Augmented Report Review, Visual Search and Querying, and Patient Imaging History Highlights. We studied these concepts with 13 radiologists and clinicians who assessed the VLM concepts as valuable, yet articulated many design considerations. Reflecting on our findings, we discuss implications for integrating VLM capabilities in radiology, and for healthcare AI more generally.


title: MVD$^2$: Efficient Multiview 3D Reconstruction for Multiview Diffusion

abstract: As a promising 3D generation technique, multiview diffusion (MVD) has received a lot of attention due to its advantages in terms of generalizability, quality, and efficiency. By finetuning pretrained large image diffusion models with 3D data, the MVD methods first generate multiple views of a 3D object based on an image or text prompt and then reconstruct 3D shapes with multiview 3D reconstruction. However, the sparse views and inconsistent details in the generated images make 3D reconstruction challenging. We present MVD$^2$, an efficient 3D reconstruction method for multiview diffusion (MVD) images. MVD$^2$ aggregates image features into a 3D feature volume by projection and convolution and then decodes volumetric features into a 3D mesh. We train MVD$^2$ with 3D shape collections and MVD images prompted by rendered views of 3D shapes. To address the discrepancy between the generated multiview images and ground-truth views of the 3D shapes, we design a simple-yet-efficient view-dependent training scheme. MVD$^2$ improves the 3D generation quality of MVD and is fast and robust to various MVD methods. After training, it can efficiently decode 3D meshes from multiview images within one second. We train MVD$^2$ with Zero-123++ and ObjectVerse-LVIS 3D dataset and demonstrate its superior performance in generating 3D models from multiview images generated by different MVD methods, using both synthetic and real images as prompts.


title: A hierarchical decomposition for explaining ML performance discrepancies

abstract: Machine learning (ML) algorithms can often differ in performance across domains. Understanding $\textit{why}$ their performance differs is crucial for determining what types of interventions (e.g., algorithmic or operational) are most effective at closing the performance gaps. Existing methods focus on $\textit{aggregate decompositions}$ of the total performance gap into the impact of a shift in the distribution of features $p(X)$ versus the impact of a shift in the conditional distribution of the outcome $p(Y|X)$; however, such coarse explanations offer only a few options for how one can close the performance gap. $\textit{Detailed variable-level decompositions}$ that quantify the importance of each variable to each term in the aggregate decomposition can provide a much deeper understanding and suggest much more targeted interventions. However, existing methods assume knowledge of the full causal graph or make strong parametric assumptions. We introduce a nonparametric hierarchical framework that provides both aggregate and detailed decompositions for explaining why the performance of an ML algorithm differs across domains, without requiring causal knowledge. We derive debiased, computationally-efficient estimators, and statistical inference procedures for asymptotically valid confidence intervals.


title: Observation of temporal topological boundary states of light in a momentum bandgap

abstract: Topological phases have prevailed across diverse disciplines, spanning electronics, photonics, and acoustics. Hitherto, the understanding of these phases has centred on energy (frequency) bandstructures, showcasing topological boundary states at spatial interfaces. Recent strides have uncovered a unique category of bandstructures characterized by gaps in momentum, referred to as momentum bandgaps or k gaps, notably driven by breakthroughs in photonic time crystals. This discovery hints at abundant topological phases defined within momentum bands, alongside a wealth of topological boundary states in the time domain. Here, we report the first experimental observation of k-gap topology in a large-scale optical temporal synthetic lattice, manifesting as temporal topological boundary states. These boundary states are uniquely situated at temporal interfaces between two subsystems with distinct k-gap topology. Counterintuitively, despite the exponential amplification of k-gap modes within both subsystems, these topological boundary states exhibit decay in both temporal directions. Our findings mark a significant pathway for delving into k gaps, temporal topological states, and time-varying physics.


title: Distributed Partial Quantum Consensus of Qubit Networks with Connected Topologies

abstract: In this paper, we consider the partial quantum consensus problem of a qubit network in a distributed view. The local quantum operation is designed based on the Hamiltonian by using the local information of each quantum system in a network of qubits. We construct the unitary transformation for each quantum system to achieve the partial quantum consensus, i.e., the directions of the quantum states in the Bloch ball will reach an agreement. A simple case of two-qubit quantum systems is considered first, and a minimum completing time of reaching partial consensus is obtained based on the geometric configuration of each qubit. Furthermore, we extend the approaches to deal with the more general N-qubit networks. Two partial quantum consensus protocols, based on the Lyapunov method for chain graphs and the geometry method for connected graphs, are proposed. The geometry method can be utilized to deal with more general connected graphs, while for the Lyapunov method, the global consensus can be obtained. The numerical simulation over a qubit network is demonstrated to verify the validity and the effectiveness of the theoretical results.


title: The semi-discrete complex modified Korteweg-de Vries equation with zero and non-zero boundary conditions: Riemann-Hilbert approach and N-soliton solutions

abstract: We focus on the semi-discrete complex modified Korteweg-de Vries (DcmKdV) equation in this paper. The direct and inverse scattering theory is developed with zero and non-zero boundary conditions (BCs) of the potential. For direct problem, the properties of the eigenfunctions and the scattering matrix, including analyticity, asymptotics and symmetries, are investigated, which facilitates the establishment of the Riemann-Hilbert (RH) problems. By solving the RH problems in the inverse problem part, the reconstruction potential formulas are obtained, which allows us to derive the N-soliton solutions in the reflectionless case. Meanwhile, the trace formulas are derived by means of studying the corresponding RH problems. Furthermore, the dynamic characteristics of the 1-soliton and 2-soliton with zero and non-zero boundary are demonstrated by graphical simulation.


title: Eagle: Ethical Dataset Given from Real Interactions

abstract: Recent studies have demonstrated that large language models (LLMs) have ethical-related problems such as social biases, lack of moral reasoning, and generation of offensive content. The existing evaluation metrics and methods to address these ethical challenges use datasets intentionally created by instructing humans to create instances including ethical problems. Therefore, the data does not reflect prompts that users actually provide when utilizing LLM services in everyday contexts. This may not lead to the development of safe LLMs that can address ethical challenges arising in real-world applications. In this paper, we create Eagle datasets extracted from real interactions between ChatGPT and users that exhibit social biases, toxicity, and immoral problems. Our experiments show that Eagle captures complementary aspects, not covered by existing datasets proposed for evaluation and mitigation of such ethical challenges. Our code is publicly available at https://huggingface.co/datasets/MasahiroKaneko/eagle.


title: Word-Sequence Entropy: Towards Uncertainty Estimation in Free-Form Medical Question Answering Applications and Beyond

abstract: Uncertainty estimation plays a pivotal role in ensuring the reliability of safety-critical human-AI interaction systems, particularly in the medical domain. However, a general method for quantifying the uncertainty of free-form answers has yet to be established in open-ended medical question-answering (QA) tasks, where irrelevant words and sequences with limited semantic information can be the primary source of uncertainty due to the presence of generative inequality. In this paper, we propose the Word-Sequence Entropy (WSE), which calibrates the uncertainty proportion at both the word and sequence levels according to the semantic relevance, with greater emphasis placed on keywords and more relevant sequences when performing uncertainty quantification. We compare WSE with 6 baseline methods on 5 free-form medical QA datasets, utilizing 7 "off-the-shelf" large language models (LLMs), and show that WSE exhibits superior performance on accurate uncertainty measurement under two standard criteria for correctness evaluation (e.g., WSE outperforms existing state-of-the-art method by 3.23% AUROC on the MedQA dataset). Additionally, in terms of the potential for real-world medical QA applications, we achieve a significant enhancement in the performance of LLMs when employing sequences with lower uncertainty, identified by WSE, as final answers (e.g., +6.36% accuracy improvement on the COVID-QA dataset), without requiring any additional task-specific fine-tuning or architectural modifications.


title: Linear Discriminant Regularized Regression

abstract: Linear Discriminant Analysis (LDA) is an important classification approach. Its simple linear form makes it easy to interpret and it is capable to handle multi-class responses. It is closely related to other classical multivariate statistical techniques, such as Fisher's discriminant analysis, canonical correlation analysis and linear regression. In this paper we strengthen its connection to multivariate response regression by characterizing the explicit relationship between the discriminant directions and the regression coefficient matrix. This key characterization leads to a new regression-based multi-class classification procedure that is flexible enough to deploy any existing structured, regularized, and even non-parametric, regression methods. Moreover, our new formulation is amenable to analysis: we establish a general strategy of analyzing the excess misclassification risk of the proposed classifier for all aforementioned regression techniques. As applications, we provide complete theoretical guarantees for using the widely used $\ell_1$-regularization as well as for using the reduced-rank regression, neither of which has yet been fully analyzed in the LDA context. Our theoretical findings are corroborated by extensive simulation studies and real data analysis.


title: Copilot Evaluation Harness: Evaluating LLM-Guided Software Programming

abstract: The integration of Large Language Models (LLMs) into Development Environments (IDEs) has become a focal point in modern software development. LLMs such as OpenAI GPT-3.5/4 and Code Llama offer the potential to significantly augment developer productivity by serving as intelligent, chat-driven programming assistants. However, utilizing LLMs out of the box is unlikely to be optimal for any given scenario. Rather, each system requires the LLM to be honed to its set of heuristics to ensure the best performance. In this paper, we introduce the Copilot evaluation harness: a set of data and tools for evaluating LLM-guided IDE interactions, covering various programming scenarios and languages. We propose our metrics as a more robust and information-dense evaluation than previous state of the art evaluation systems. We design and compute both static and execution based success metrics for scenarios encompassing a wide range of developer tasks, including code generation from natural language (generate), documentation generation from code (doc), test case generation (test), bug-fixing (fix), and workspace understanding and query resolution (workspace). These success metrics are designed to evaluate the performance of LLMs within a given IDE and its respective parameter space. Our learnings from evaluating three common LLMs using these metrics can inform the development and validation of future scenarios in LLM guided IDEs.


title: Color-flavor locked strange stars with a mirror-dark-matter core and the observations of compact stars

abstract: We investigate the structure and the tidal deformability of the color-flavor locked strange stars with a mirror-dark-matter core. Assuming the stars in the GW170817 event have a mirror-dark-matter core, the observations of the central compact object within the supernova remnant HESS J1731-347 and the compact objects in the GW190814 and GW170817 events could be explained simultaneously with a pairing gap much smaller than 200 MeV, while a pairing gap larger than about 200 MeV must be employed without the consideration of a mirror-dark-matter core. More importantly, we find that for the case of the quartic coefficient $a_{4}< 0.589$, if the mirror-dark-matter core in the compact stars of GW170817 is large enough (e.g., the mass fraction of the mirror-dark-matter core is larger than 22.8% for $a_{4}= 0.55$), the minimum allowed value of the pairing gap could be less than 46.5 MeV (i.e., one half of the value of the strange quark mass which is taken as 93 MeV in this paper), which leads to the result that all astrophysical observations mentioned above could be satisfied without violating the conformal bound or the recently proposed positive trace anomally bound.


title: Innovation Diffusion in EV Charging Location Decisions: Integrating Demand & Supply through Market Dynamics

abstract: This paper offers a strategic approach to Electric Vehicles (EVs) charging network planning, emphasizing the integration of demand and supply dynamics via continuous-time fluid queue models and discrete flow refueling location modeling, all in the context of innovation diffusion principles. We employ a continuous-time approximation based on Ordinary Differential Equations (ODEs) to design multi-year supply curves, a method that stands in contrast to conventional practices which often overlook inter-year transitions and ongoing processes. For medium-term charging station location planning (CSLP), we apply a flow refueling location model (FRLM) within grid-based multi-level networks, considering both multiple-path networks and capacity constraints. The grid-based network planning strategy uses a three-tier (Macro-Meso-Micro) approach for thorough EV charging station placement, with the macro-level covering entire cities, the meso-level assessing detailed EV routes and bridging the macro to micro levels, and the micro-level focusing on precise station placement for accessibility and efficiency. Our investigation into overutilization and underutilization scenarios delivers valuable insights for policymaking and cost-benefit analyses. Illustrating our approach with the example of the Chicago sketch network, we introduce an integrated demand-supply model suitable for a single region and extendable to multiple regions, thereby addressing a gap in the existing literature. Our proposed methodology focuses on EV station placement, taking into account future needs, geographical capacities, and the importance of scenario analysis, which empowers strategic resource planning for EV charging networks over extended timeframes, thus aiding the transition towards a more sustainable and efficient transportation system.


title: Structure-agnostic Optimality of Doubly Robust Learning for Treatment Effect Estimation

abstract: Average treatment effect estimation is the most central problem in causal inference with application to numerous disciplines. While many estimation strategies have been proposed in the literature, the statistical optimality of these methods has still remained an open area of investigation, especially in regimes where these methods do not achieve parametric rates. In this paper, we adopt the recently introduced structure-agnostic framework of statistical lower bounds, which poses no structural properties on the nuisance functions other than access to black-box estimators that achieve some statistical estimation rate. This framework is particularly appealing when one is only willing to consider estimation strategies that use non-parametric regression and classification oracles as black-box sub-processes. Within this framework, we prove the statistical optimality of the celebrated and widely used doubly robust estimators for both the Average Treatment Effect (ATE) and the Average Treatment Effect on the Treated (ATT), as well as weighted variants of the former, which arise in policy evaluation.

categories: stat.ML cs.LG econ.EM math.ST stat.ME stat.TH


title: Quasinormal modes of a charged black hole with scalar hair

abstract: From a five-dimensional Einstein-Maxwell theory, Bah et al. constructed a singularity free topology star/black hole [Phys. Rev. Lett. 126, 151101 (2021)]. After the Klein-Kluza reduction, i.e., integrating the extra space dimension, it can obtain an effective four-dimensional static spherical charged black hole with scalar hair. In this paper, we study the quasinormal modes (QNMs) of the scalar field, electromagnetic field, and gravitational field on the background of this effective four-dimensional charged black hole. The radial parts of the perturbed fields all satisfy a Schr\"{o}dinger-like equation. Using the asymptotic iteration method, we obtain the QNM frequencies semianalytically. For low overtone QNMs, the results obtained from the asymptotic iteration method and the Wentzel-Kramers-Brillouin approximation method agree well. In the null coordinates, the evolution of a Gaussian package is also studied. The QNM frequencies obtained by fitting the evolution data also agree well with the results obtained by the asymptotic iteration method.


title: Efficient Solvers for Wyner Common Information with Application to Multi-Modal Clustering

abstract: We propose two novel extensions of the Wyner common information optimization problem. Each relaxes one fundamental constraints in Wyner's formulation. The \textit{Variational Wyner Common Information} relaxes the matching constraint to the known distribution while imposing conditional independence to the feasible solution set. We derive a tight surrogate upper bound of the obtained unconstrained Lagrangian via the theory of variational inference, which can be minimized efficiently. Our solver caters to problems where conditional independence holds with significantly reduced computation complexity; On the other hand, the \textit{Bipartite Wyner Common Information} relaxes the conditional independence constraint whereas the matching condition is enforced on the feasible set. By leveraging the difference-of-convex structure of the formulated optimization problem, we show that our solver is resilient to conditional dependent sources. Both solvers are provably convergent (local stationary points), and empirically, they obtain more accurate solutions to Wyner's formulation with substantially less runtime. Moreover, them can be extended to unknown distribution settings by parameterizing the common randomness as a member of the exponential family of distributions. Our approaches apply to multi-modal clustering problems, where multiple modalities of observations come from the same cluster. Empirically, our solvers outperform the state-of-the-art multi-modal clustering algorithms with significantly improved performance.


title: Asymmetric Relaxations Through the Lens of Information Geometry

abstract: We introduce a novel perspective on Newton's Law of Cooling by framing it within the context of information geometry. By characterizing it as a specific representation of a gradient flow, we establish a connection with the well-known Horse-Carrot Theorem and reveal previously unexplored instances of asymmetric relaxations in the context of endoreversible processes. We present a completely general and practical criterion for predicting asymmetries based on the Amari-Chentsov tensor, whose applicability extends to both classical and quantum thermodynamics. Illustrative examples include faster cooling down of quantum ideal gases, and relaxations reminiscent of the Mpemba effect in classical ideal gases.


title: Can Large Language Models Detect Misinformation in Scientific News Reporting?

abstract: Scientific facts are often spun in the popular press with the intent to influence public opinion and action, as was evidenced during the COVID-19 pandemic. Automatic detection of misinformation in the scientific domain is challenging because of the distinct styles of writing in these two media types and is still in its nascence. Most research on the validity of scientific reporting treats this problem as a claim verification challenge. In doing so, significant expert human effort is required to generate appropriate claims. Our solution bypasses this step and addresses a more real-world scenario where such explicit, labeled claims may not be available. The central research question of this paper is whether it is possible to use large language models (LLMs) to detect misinformation in scientific reporting. To this end, we first present a new labeled dataset SciNews, containing 2.4k scientific news stories drawn from trusted and untrustworthy sources, paired with related abstracts from the CORD-19 database. Our dataset includes both human-written and LLM-generated news articles, making it more comprehensive in terms of capturing the growing trend of using LLMs to generate popular press articles. Then, we identify dimensions of scientific validity in science news articles and explore how this can be integrated into the automated detection of scientific misinformation. We propose several baseline architectures using LLMs to automatically detect false representations of scientific findings in the popular press. For each of these architectures, we use several prompt engineering strategies including zero-shot, few-shot, and chain-of-thought prompting. We also test these architectures and prompting strategies on GPT-3.5, GPT-4, and Llama2-7B, Llama2-13B.


title: Optimal Mechanism in a Dynamic Stochastic Knapsack Environment

abstract: This study introduces an optimal mechanism in a dynamic stochastic knapsack environment. The model features a single seller who has a fixed quantity of a perfectly divisible item. Impatient buyers with a piece-wise linear utility function arrive randomly and they report the two-dimensional private information: marginal value and demanded quantity. We derive a revenue-maximizing dynamic mechanism in a finite discrete time framework that satisfies incentive compatibility, individual rationality, and feasibility conditions. It is achieved by characterizing buyers' utility and deriving the Bellman equation. Moreover, we propose the essential penalty scheme for incentive compatibility, as well as the allocation and payment policies. Lastly, we propose algorithms to approximate the optimal policy, based on the Monte Carlo simulation-based regression method and reinforcement learning.


title: Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization

abstract: In the rapidly advancing arena of large language models (LLMs), a key challenge is to enhance their capabilities amid a looming shortage of high-quality training data. Our study starts from an empirical strategy for the light continual training of LLMs using their original pre-training data sets, with a specific focus on selective retention of samples that incur moderately high losses. These samples are deemed informative and beneficial for model refinement, contrasting with the highest-loss samples, which would be discarded due to their correlation with data noise and complexity. We then formalize this strategy into a principled framework of Instance-Reweighted Distributionally Robust Optimization (IR-DRO). IR-DRO is designed to dynamically prioritize the training focus on informative samples through an instance reweighting mechanism, streamlined by a closed-form solution for straightforward integration into established training protocols. Through rigorous experimentation with various models and datasets, our findings indicate that our sample-targeted methods significantly improve LLM performance across multiple benchmarks, in both continual pre-training and instruction tuning scenarios. Our codes are available at https://github.com/VITA-Group/HardFocusTraining.


title: Hyers-Ulam stability of the first order difference equation with average growth rate

abstract: The first order difference equation induced by the sequence of maps on $ \mathbb{C} $ has Hyers-Ulam stability where the limit of the geometric average of growth rate is convergent and not equal to one. %The average growth rate is a generalization of contracting or expanding constant of maps. We show no Hyers-Ulam stability where the average growth rate is (pre)periodic even though each periodic growth rate is strictly less than one. Examples of difference equation generated by time dependent maps which contains contracting maps and expanding maps are given.


title: Qsnail: A Questionnaire Dataset for Sequential Question Generation

abstract: The questionnaire is a professional research methodology used for both qualitative and quantitative analysis of human opinions, preferences, attitudes, and behaviors. However, designing and evaluating questionnaires demands significant effort due to their intricate and complex structure. Questionnaires entail a series of questions that must conform to intricate constraints involving the questions, options, and overall structure. Specifically, the questions should be relevant and specific to the given research topic and intent. The options should be tailored to the questions, ensuring they are mutually exclusive, completed, and ordered sensibly. Moreover, the sequence of questions should follow a logical order, grouping similar topics together. As a result, automatically generating questionnaires presents a significant challenge and this area has received limited attention primarily due to the scarcity of high-quality datasets. To address these issues, we present Qsnail, the first dataset specifically constructed for the questionnaire generation task, which comprises 13,168 human-written questionnaires gathered from online platforms. We further conduct experiments on Qsnail, and the results reveal that retrieval models and traditional generative models do not fully align with the given research topic and intents. Large language models, while more closely related to the research topic and intents, exhibit significant limitations in terms of diversity and specificity. Despite enhancements through the chain-of-thought prompt and finetuning, questionnaires generated by language models still fall short of human-written questionnaires. Therefore, questionnaire generation is challenging and needs to be further explored. The dataset is available at: https://github.com/LeiyanGithub/qsnail.


title: Can Language Models Act as Knowledge Bases at Scale?

abstract: Large language models (LLMs) have demonstrated remarkable proficiency in understanding and generating responses to complex queries through large-scale pre-training. However, the efficacy of these models in memorizing and reasoning among large-scale structured knowledge, especially world knowledge that explicitly covers abundant factual information remains questionable. Addressing this gap, our research investigates whether LLMs can effectively store, recall, and reason with knowledge on a large scale comparable to latest knowledge bases (KBs) such as Wikidata. Specifically, we focus on three crucial aspects to study the viability: (1) the efficiency of LLMs with different sizes in memorizing the exact knowledge in the large-scale KB; (2) the flexibility of recalling the memorized knowledge in response to natural language queries; (3) the capability to infer new knowledge through reasoning. Our findings indicate that while LLMs hold promise as large-scale KBs capable of retrieving and responding with flexibility, enhancements in their reasoning capabilities are necessary to fully realize their potential.