title: Charting the Realms of Mesoscale Cloud Organisation using Unsupervised Learning

abstract: Quantifying the driving mechanisms and effect on Earth's energy budget, of mesoscale shallow cloud organisation, remains difficult. Partly because quantifying the atmosphere's organisational state through objective means remains challenging. We present the first map of the full continuum of convective organisation states by extracting the manifold within an unsupervised neural networks's internal representation. On the manifold distinct organisational regimes, defined in prior work, sit as waymarkers in this continuum. Composition of reanalysis and observations onto the manifold, shows wind-speed and water vapour concentration as key environmental characteristics varying with organisation. We show, for the first time, that mesoscale shallow cloud organisation produces $\pm 1.4\%$ variations in albedo in addition to variations from cloud-fraction changes alone. We further demonstrate how the manifold's continuum representation captures the temporal evolution of organisation. By enabling study of states and transitions in organisation (in simulations and observations) the presented technique paves the way for better representation of shallow clouds in simulations of Earth's future climate.


title: Denoising Diffusion Probabilistic Models for Hardware-Impaired Communications

abstract: Generative AI has received significant attention among a spectrum of diverse industrial and academic domains, thanks to the magnificent results achieved from deep generative models such as generative pre-trained transformers (GPT) and diffusion models. In this paper, we explore the applications of denoising diffusion probabilistic models (DDPMs) in wireless communication systems under practical assumptions such as hardware impairments (HWI), low-SNR regime, and quantization error. Diffusion models are a new class of state-of-the-art generative models that have already showcased notable success with some of the popular examples by OpenAI1 and Google Brain2. The intuition behind DDPM is to decompose the data generation process over small ``denoising'' steps. Inspired by this, we propose using denoising diffusion model-based receiver for a practical wireless communication scheme, while providing network resilience in low-SNR regimes, non-Gaussian noise, different HWI levels, and quantization error. We evaluate the reconstruction performance of our scheme in terms of mean-squared error (MSE) metric. Our results show that more than 25 dB improvement in MSE is achieved compared to deep neural network (DNN)-based receivers. We also highlight robust out-of-distribution performance under non-Gaussian noise.


title: Local Differential Privacy in Graph Neural Networks: a Reconstruction Approach

abstract: Graph Neural Networks have achieved tremendous success in modeling complex graph data in a variety of applications. However, there are limited studies investigating privacy protection in GNNs. In this work, we propose a learning framework that can provide node privacy at the user level, while incurring low utility loss. We focus on a decentralized notion of Differential Privacy, namely Local Differential Privacy, and apply randomization mechanisms to perturb both feature and label data at the node level before the data is collected by a central server for model training. Specifically, we investigate the application of randomization mechanisms in high-dimensional feature settings and propose an LDP protocol with strict privacy guarantees. Based on frequency estimation in statistical analysis of randomized data, we develop reconstruction methods to approximate features and labels from perturbed data. We also formulate this learning framework to utilize frequency estimates of graph clusters to supervise the training procedure at a sub-graph level. Extensive experiments on real-world and semi-synthetic datasets demonstrate the validity of our proposed model.


title: Neural Network Driven, Interactive Design for Nonlinear Optical Molecules Based on Group Contribution Method

abstract: A Lewis-mode group contribution method (LGC) -- multi-stage Bayesian neural network (msBNN) -- evolutionary algorithm (EA) framework is reported for rational design of D-Pi-A type organic small-molecule nonlinear optical materials is presented. Upon combination of msBNN and corrected Lewis-mode group contribution method (cLGC), different optical properties of molecules are afforded accurately and efficiently - by using only a small data set for training. Moreover, by employing the EA model designed specifically for LGC, structural search is well achievable. The logical origins of the well performance of the framework are discussed in detail. Considering that such a theory guided, machine learning framework combines chemical principles and data-driven tools, most likely, it will be proven efficient to solve molecular design related problems in wider fields.


title: A Bayesian Approach to Robust Inverse Reinforcement Learning

abstract: We consider a Bayesian approach to offline model-based inverse reinforcement learning (IRL). The proposed framework differs from existing offline model-based IRL approaches by performing simultaneous estimation of the expert's reward function and subjective model of environment dynamics. We make use of a class of prior distributions which parameterizes how accurate the expert's model of the environment is to develop efficient algorithms to estimate the expert's reward and subjective dynamics in high-dimensional settings. Our analysis reveals a novel insight that the estimated policy exhibits robust performance when the expert is believed (a priori) to have a highly accurate model of the environment. We verify this observation in the MuJoCo environments and show that our algorithms outperform state-of-the-art offline IRL algorithms.


title: Simulating Neutral Atom Quantum Systems with Tensor Network States

abstract: In this paper, we describe a tensor network simulation of a neutral atom quantum system under the presence of noise, while introducing a new purity-preserving truncation technique that compromises between the simplicity of the matrix product state and the positivity of the matrix product density operator. We apply this simulation to a near-optimized iteration of the quantum approximate optimization algorithm on a transverse field Ising model in order to investigate the influence of large system sizes on the performance of the algorithm. We find that while circuits with a large number of qubits fail more often under noise that depletes the qubit population, their outputs on a successful measurement are just as robust under Rydberg atom dissipation or qubit dephasing as smaller systems. However, such circuits might not perform as well under coherent multi-qubit errors such as Rydberg atom crosstalk. We also find that the optimized parameters are especially robust to noise, suggesting that a noisier quantum system can be used to find the optimal parameters before switching to a cleaner system for measurements of observables.


title: Casteist but Not Racist? Quantifying Disparities in Large Language Model Bias between India and the West

abstract: Large Language Models (LLMs), now used daily by millions of users, can encode societal biases, exposing their users to representational harms. A large body of scholarship on LLM bias exists but it predominantly adopts a Western-centric frame and attends comparatively less to bias levels and potential harms in the Global South. In this paper, we quantify stereotypical bias in popular LLMs according to an Indian-centric frame and compare bias levels between the Indian and Western contexts. To do this, we develop a novel dataset which we call Indian-BhED (Indian Bias Evaluation Dataset), containing stereotypical and anti-stereotypical examples for caste and religion contexts. We find that the majority of LLMs tested are strongly biased towards stereotypes in the Indian context, especially as compared to the Western context. We finally investigate Instruction Prompting as a simple intervention to mitigate such bias and find that it significantly reduces both stereotypical and anti-stereotypical biases in the majority of cases for GPT-3.5. The findings of this work highlight the need for including more diverse voices when evaluating LLMs.


title: DP-PQD: Privately Detecting Per-Query Gaps In Synthetic Data Generated By Black-Box Mechanisms

abstract: Synthetic data generation methods, and in particular, private synthetic data generation methods, are gaining popularity as a means to make copies of sensitive databases that can be shared widely for research and data analysis. Some of the fundamental operations in data analysis include analyzing aggregated statistics, e.g., count, sum, or median, on a subset of data satisfying some conditions. When synthetic data is generated, users may be interested in knowing if their aggregated queries generating such statistics can be reliably answered on the synthetic data, for instance, to decide if the synthetic data is suitable for specific tasks. However, the standard data generation systems do not provide "per-query" quality guarantees on the synthetic data, and the users have no way of knowing how much the aggregated statistics on the synthetic data can be trusted. To address this problem, we present a novel framework named DP-PQD (differentially-private per-query decider) to detect if the query answers on the private and synthetic datasets are within a user-specified threshold of each other while guaranteeing differential privacy. We give a suite of private algorithms for per-query deciders for count, sum, and median queries, analyze their properties, and evaluate them experimentally.


title: A pre-metric formulation of the Fresnel wave and ray surfaces

abstract: The conventional formulation of the Fresnel wave and ray surfaces typically involves the implicit use of Euclidian metrics in order to justify treating vectors and covectors as indistinguishable. This tends to disguise the fact that one of the surfaces lives in the cotangent spaces to the spatial (or space-time) manifold, while the other lives in the tangent spaces. Moreover, no mention is made of how to get from tangent spaces to cotangent spaces in the absence of a metric. The following analysis shows how to resolve those issues within the framework of pre-metric electromagnetism. The way that electromagnetic waves imply spatial frames, coframes, and metrics will be explained.


title: Norm Growth, Non-uniqueness, and Anomalous Dissipation in Passive Scalars

abstract: We construct a divergence-free velocity field $u:[0,T] \times \mathbb{T}^2 \to \mathbb{R}^2$ satisfying $$u \in C^\infty([0,T];C^\alpha(\mathbb{T}^2)) \quad \forall \alpha \in [0,1)$$ such that the corresponding drift-diffusion equation exhibits anomalous dissipation for every smooth initial data. We also show that, given any $\alpha_0 < 1$, the flow can be modified such that it is uniformly bounded only in $C^{\alpha_0}(\mathbb{T}^2)$ and the regularity of solutions satisfy sharp (time-integrated) bounds predicted by the Obukhov-Corrsin theory. The proof is based on a general principle implying $H^1$ growth for all solutions to the transport equation, which may be of independent interest.


title: Lamination-based efficient treatment of weak discontinuities for non-conforming finite element meshes

abstract: When modelling discontinuities (interfaces) using the finite element method, the standard approach is to use a conforming finite-element mesh in which the mesh matches the interfaces. However, this approach can prove cumbersome if the geometry is complex, in particular in 3D. In this work, we develop an efficient technique for a non-conforming finite-element treatment of weak discontinuities by using laminated microstructures. The approach is inspired by the so-called composite voxel technique that has been developed for FFT-based spectral solvers in computational homogenization. The idea behind the method is rather simple. Each finite element that is cut by an interface is treated as a simple laminate with the volume fraction of the phases and the lamination orientation determined in terms of the actual geometrical arrangement of the interface within the element. The approach is illustrated by several computational examples relevant to the micromechanics of heterogeneous materials. Elastic and elastic-plastic materials at small and finite strain are considered in the examples. The performance of the proposed method is compared to two alternative, simple methods showing that the new approach is in most cases superior to them while maintaining the simplicity.


title: Cutting corners to suppress high-order modes in Mie resonator arrays

abstract: Mie resonators as lattice resonant metasurfaces have the capability to produce structural colour. However, design criteria for these metasurfaces are still being investigated. In this work, we numerically examine how the two-dimensional nanostructure shape in a lattice array affects the colorimetric response of the metasurface under linearly polarised light excitation. First, the transformation from a square-shaped to rectangle-shaped nanostructure array resulted in polarisation-sensitive metasurfaces with colorimetric outputs bound along a line on the CIE 1931 2-degree Standard Observer colour space. The bounds of the colorimetry line were tuneable to any desired chromatic range. Second, the removal of the corners in square- or rectangle-shaped nanostructures to create t-shaped nanostructure arrays displayed a dampening effect on the high-order resonance. Finally, we analytically determined that the colour saturation could increase when moving from rectangle-shaped to t-shaped nanostructure arrays. From these results, we present two design guidelines for lattice resonant metasurfaces: (1) Constructing the nanostructure to support fundamental resonances at different wavelengths enables two-colour-bound movement when excited by successive angles of linearly polarised light; (2) Removing portions of the nanostructure that only support high-order resonances dampens these modes while maintaining support for fundamental resonances. These results present first-principles guidelines for engineering nanoparticles in lattice resonant metasurfaces, offering a new toolbox for polarised-light sensing and colorimetric applications.


title: Polytopal composite finite elements for modeling concrete fracture based on nonlocal damage models

abstract: The paper presents an assumed strain formulation over polygonal meshes to accurately evaluate the strain fields in nonlocal damage models. An assume strained technique based on the Hu-Washizu variational principle is employed to generate a new strain approximation instead of direct derivation from the basis functions and the displacement fields. The underlying idea embedded in arbitrary finite polygons is named as Polytopal composite finite elements (PCFEM). The PCFEM is accordingly applied within the framework of the nonlocal model of continuum damage mechanics to enhance the description of damage behaviours in which highly localized deformations must be captured accurately. This application is helpful to reduce the mesh-sensitivity and elaborate the process-zone of damage models. Several numerical examples are designed for various cases of fracture to discuss and validate the computational capability of the present method through comparison with published numerical results and experimental data from the literature.


title: Automated Characterization and Monitoring of Material Shape using Riemannian Geometry

abstract: Shape affects both the physical and chemical properties of a material. Characterizing the roughness, convexity, and general geometry of a material can yield information on its catalytic efficiency, solubility, elasticity, porosity, and overall effectiveness in the application of interest. However, material shape can be defined in a multitude of conflicting ways where different aspects of a material's geometry are emphasized over others, leading to bespoke measures of shape that are not easily generalizable. In this paper, we explore the use of Riemannian geometry in the analysis of shape and show that a Riemannian geometric framework for shape analysis is generalizable, computationally scalable, and can be directly integrated into common data analysis methods. In this framework, material shapes are abstracted as points on a Riemannian manifold. This information can be used to construct statistical moments (e.g., means, variances) and perform tasks such as dimensionality reduction and statistical process control. We provide a practical introduction to the mathematics of shape analysis through Riemannian geometry and illustrate its application on a manufactured/mined granular material dataset provided by Covia Corp. We show that the Riemannian framework can be used to automatically extract and quantify the shape of granular materials in a statistically rigorous manner.


title: Dependent Type Refinements for Futures

abstract: Type refinements combine the compositionality of typechecking with the expressivity of program logics, offering a synergistic approach to program verification. In this paper we apply dependent type refinements to SAX, a futures-based process calculus that arises from the Curry-Howard interpretation of the intuitionistic semi-axiomatic sequent calculus and includes unrestricted recursion both at the level of types and processes. With our type refinement system, we can reason about the partial correctness of SAX programs, complementing prior work on sized type refinements that supports reasoning about termination. Our design regime synthesizes the infinitary proof theory of SAX with that of bidirectional typing and Hoare logic, deriving some standard reasoning principles for data and (co)recursion while enabling information hiding for codata. We prove syntactic type soundness, which entails a notion of partial correctness that respects codata encapsulation. We illustrate our language through a few simple examples.


title: Gate-tunable Superconductivity in Hybrid InSb-Pb Nanowires

abstract: We present a report on hybrid InSb-Pb nanowires that combine high spin-orbit coupling with a high critical field and a large superconducting gap. Material characterization indicates the Pb layer of high crystal quality on the nanowire side facets. Hard induced superconducting gaps and gate-tunable supercurrent are observed in the hybrid nanowires. These results showcase the promising potential of this material combination for a diverse range of applications in hybrid quantum transport devices.

categories: cond-mat.mes-hall cond-mat.mtrl-sci cond-mat.supr-con quant-ph


title: ICLEF: In-Context Learning with Expert Feedback for Explainable Style Transfer

abstract: While state-of-the-art large language models (LLMs) can excel at adapting text from one style to another, current work does not address the explainability of style transfer models. Recent work has explored generating textual explanations from larger teacher models and distilling them into smaller student models. One challenge with such approach is that LLM outputs may contain errors that require expertise to correct, but gathering and incorporating expert feedback is difficult due to cost and availability. To address this challenge, we propose ICLEF, a novel human-AI collaboration approach to model distillation that incorporates scarce expert human feedback by combining in-context learning and model self-critique. We show that our method leads to generation of high-quality synthetic explainable style transfer datasets for formality (e-GYAFC) and subjective bias (e-WNC). Via automatic and human evaluation, we show that specialized student models fine-tuned on our datasets outperform generalist teacher models on the explainable style transfer task in one-shot settings, and perform competitively compared to few-shot teacher models, highlighting the quality of the data and the role of expert feedback. In an extrinsic task of authorship attribution, we show that explanations generated by smaller models fine-tuned on e-GYAFC are more predictive of authorship than explanations generated by few-shot teacher models.


title: Phase field method for quasi-static hydro-fracture in porous media under stress boundary condition considering the effect of initial stress field

abstract: Phase field model (PFM) is an efficient fracture modeling method and has high potential for hydraulic fracturing (HF). However, the current PFMs in HF do not consider well the effect of in-situ stress field and the numerical examples of porous media with stress boundary conditions were rarely presented. The main reason is that if the remote stress is applied on the boundaries of the calculation domain, there will be relatively large deformation induced on these stress boundaries, which is not consistent with the engineering observations. To eliminate this limitation, this paper proposes a new phase field method to describe quasi-static hydraulic fracture propagation in porous media subjected to stress boundary conditions, and the new method is more in line with engineering practice. A new energy functional, which considers the effect of initial in-situ stress field, is established and then it is used to achieve the governing equations for the displacement and phase fields through the variational approach. Biot poroelasticity theory is used to couple the fluid pressure field and the displacement field while the phase field is used for determining the fluid properties from the intact domain to the fully broken domain. In addition, we present several 2D and 3D examples to show the effects of in-situ stress on hydraulic fracture propagation. The numerical examples indicate that under stress boundary condition our approach obtains correct displacement distribution and it is capable of capturing complex hydraulic fracture growth patterns.


title: Viewpoint Integration and Registration with Vision Language Foundation Model for Image Change Understanding

abstract: Recently, the development of pre-trained vision language foundation models (VLFMs) has led to remarkable performance in many tasks. However, these models tend to have strong single-image understanding capability but lack the ability to understand multiple images. Therefore, they cannot be directly applied to cope with image change understanding (ICU), which requires models to capture actual changes between multiple images and describe them in language. In this paper, we discover that existing VLFMs perform poorly when applied directly to ICU because of the following problems: (1) VLFMs generally learn the global representation of a single image, while ICU requires capturing nuances between multiple images. (2) The ICU performance of VLFMs is significantly affected by viewpoint variations, which is caused by the altered relationships between objects when viewpoint changes. To address these problems, we propose a Viewpoint Integration and Registration method. Concretely, we introduce a fused adapter image encoder that fine-tunes pre-trained encoders by inserting designed trainable adapters and fused adapters, to effectively capture nuances between image pairs. Additionally, a viewpoint registration flow and a semantic emphasizing module are designed to reduce the performance degradation caused by viewpoint variations in the visual and semantic space, respectively. Experimental results on CLEVR-Change and Spot-the-Diff demonstrate that our method achieves state-of-the-art performance in all metrics.


title: Replacing softmax with ReLU in Vision Transformers

abstract: Previous research observed accuracy degradation when replacing the attention softmax with a point-wise activation such as ReLU. In the context of vision transformers, we find that this degradation is mitigated when dividing by sequence length. Our experiments training small to large vision transformers on ImageNet-21k indicate that ReLU-attention can approach or match the performance of softmax-attention in terms of scaling behavior as a function of compute.


abstract: To make effective decisions in novel environments with long-horizon goals, it is crucial to engage in hierarchical reasoning across spatial and temporal scales. This entails planning abstract subgoal sequences, visually reasoning about the underlying plans, and executing actions in accordance with the devised plan through visual-motor control. We propose Compositional Foundation Models for Hierarchical Planning (HiP), a foundation model which leverages multiple expert foundation model trained on language, vision and action data individually jointly together to solve long-horizon tasks. We use a large language model to construct symbolic plans that are grounded in the environment through a large video diffusion model. Generated video plans are then grounded to visual-motor control, through an inverse dynamics model that infers actions from generated videos. To enable effective reasoning within this hierarchy, we enforce consistency between the models via iterative refinement. We illustrate the efficacy and adaptability of our approach in three different long-horizon table-top manipulation tasks.


title: Robust Frame-to-Frame Camera Rotation Estimation in Crowded Scenes

abstract: We present an approach to estimating camera rotation in crowded, real-world scenes from handheld monocular video. While camera rotation estimation is a well-studied problem, no previous methods exhibit both high accuracy and acceptable speed in this setting. Because the setting is not addressed well by other datasets, we provide a new dataset and benchmark, with high-accuracy, rigorously verified ground truth, on 17 video sequences. Methods developed for wide baseline stereo (e.g., 5-point methods) perform poorly on monocular video. On the other hand, methods used in autonomous driving (e.g., SLAM) leverage specific sensor setups, specific motion models, or local optimization strategies (lagging batch processing) and do not generalize well to handheld video. Finally, for dynamic scenes, commonly used robustification techniques like RANSAC require large numbers of iterations, and become prohibitively slow. We introduce a novel generalization of the Hough transform on SO(3) to efficiently and robustly find the camera rotation most compatible with optical flow. Among comparably fast methods, ours reduces error by almost 50\% over the next best, and is more accurate than any method, irrespective of speed. This represents a strong new performance point for crowded scenes, an important setting for computer vision. The code and the dataset are available at https://fabiendelattre.com/robust-rotation-estimation.


title: Chain-of-Thought Reasoning is a Policy Improvement Operator

abstract: Large language models have astounded the world with fascinating new capabilities. However, they currently lack the ability to teach themselves new skills, relying instead on large amounts of human-generated training data. We introduce SECToR (Self-Education via Chain-of-Thought Reasoning), a proof-of-concept demonstration that language models can teach themselves new skills using chain-of-thought reasoning. During the self-learning loop, SECToR asks models to solve addition problems using chain-of-thought reasoning before training the next version of the model to solve those same problems directly without using such reasoning. This process often results in an improved model which can, when again augmented with chain-of-thought reasoning, solve even harder problems than the original model, allowing the self-learning loop to continue. Language models trained via SECToR autonomously learn to add up to the longest-length-digit numbers without access to any ground truth examples beyond an initial supervised fine-tuning phase consisting only of numbers with 6 or fewer digits. Our central hypothesis is that chain-of-thought reasoning can act as a policy improvement operator, similarly to how Monte-Carlo Tree Search is used in AlphaZero (Silver et al., 2017). We hope that this research can lead to new directions in which language models can learn to teach themselves without the need for human demonstrations.


title: Neural Machine Translation Models Can Learn to be Few-shot Learners

abstract: The emergent ability of Large Language Models to use a small number of examples to learn to perform in novel domains and tasks, also called in-context learning (ICL). In this work, we show that a much smaller model can be trained to perform ICL by fine-tuning towards a specialized training objective, exemplified on the task of domain adaptation for neural machine translation. With this capacity for ICL, the model can take advantage of relevant few-shot examples to adapt its output towards the domain. We compare the quality of this domain adaptation to traditional supervised techniques and ICL with a 40B-parameter Large Language Model. Our approach allows efficient batch inference on a mix of domains and outperforms state-of-the-art baselines in terms of both translation quality and immediate adaptation rate, i.e. the ability to reproduce a specific term after being shown a single example.


title: Are Multilingual LLMs Culturally-Diverse Reasoners? An Investigation into Multicultural Proverbs and Sayings

abstract: Large language models (LLMs) are highly adept at question answering and reasoning tasks, but when reasoning in a situational context, human expectations vary depending on the relevant cultural common ground. As languages are associated with diverse cultures, LLMs should also be culturally-diverse reasoners. In this paper, we study the ability of a wide range of state-of-the-art multilingual LLMs (mLLMs) to reason with proverbs and sayings in a conversational context. Our experiments reveal that: (1) mLLMs "know" limited proverbs and memorizing proverbs does not mean understanding them within a conversational context; (2) mLLMs struggle to reason with figurative proverbs and sayings, and when asked to select the wrong answer (instead of asking it to select the correct answer); and (3) there is a "culture gap" in mLLMs when reasoning about proverbs and sayings translated from other languages. We construct and release our evaluation dataset MAPS (MulticultrAl Proverbs and Sayings) for proverb understanding with conversational context for six different languages.


title: Confirmation of an anomalously low dark matter content for the galaxy NGC1052-DF4 from deep, high resolution continuum spectroscopy

abstract: NGC1052-DF4 was found to be the second "galaxy lacking dark matter" in the NGC1052 group, based on its velocity dispersion of $\sigma_{\rm gc}=4.2^{+4.4}_{-2.2}$ km/s as measured from the radial velocities of seven of its globular clusters. Here we verify this result by measuring the stellar velocity dispersion of the galaxy. We observed the diffuse stellar light in NGC1052-DF4 with the Keck Cosmic Web Imager (KCWI) in its highest resolution mode, with $\sigma_{\mathrm{instr}}\approx 7$ km/s. With a total science + sky exposure time of 34hrs, the resulting spectrum is exceptional both in its spectral resolution and its S/N ratio of 23\r{A}$^{-1}$. We find a stellar velocity dispersion of $\sigma_{\rm stars} = 8.0^{+2.3}_{-1.9}$ km/s, consistent with the previous measurement from the globular clusters. Combining both measurements gives a fiducial dispersion of $\sigma_{\rm f} = 6.3_{-1.6}^{+2.5}$ km/s. The implied dynamical mass within the half-light radius is $8_{-4}^{+6} \times 10^7 M_{\odot}$. The expected velocity dispersion of NGC1052-DF4 from the stellar mass alone is $7 \pm 1$ km/s, and for an NFW halo that follows the stellar mass -- halo mass relation and the halo mass -- concentration relation, the expectation is $\sim 30$ km/s. The low velocity dispersion rules out a normal NFW dark matter halo, and we confirm that NGC1052-DF4 is one of at least two galaxies in the NGC1052 group that have an anomalously low dark matter content. While any viable model for their formation should explain the properties of both galaxies, we note that NGC1052-DF4 now poses the largest challenge as it has the most stringent constraints on its dynamical mass.


title: Attention-Only Transformers and Implementing MLPs with Attention Heads

abstract: The transformer architecture is widely used in machine learning models and consists of two alternating sublayers: attention heads and MLPs. We prove that an MLP neuron can be implemented by a masked attention head with internal dimension 1 so long as the MLP's activation function comes from a restricted class including SiLU and close approximations of ReLU and GeLU. This allows one to convert an MLP-and-attention transformer into an attention-only transformer at the cost of greatly increasing the number of attention heads. We also prove that attention heads can perform the components of an MLP (linear transformations and activation functions) separately. Finally, we prove that attention heads can encode arbitrary masking patterns in their weight matrices to within arbitrarily small error.


title: "Merge Conflicts!" Exploring the Impacts of External Distractors to Parametric Knowledge Graphs

abstract: Large language models (LLMs) acquire extensive knowledge during pre-training, known as their parametric knowledge. However, in order to remain up-to-date and align with human instructions, LLMs inevitably require external knowledge during their interactions with users. This raises a crucial question: How will LLMs respond when external knowledge interferes with their parametric knowledge? To investigate this question, we propose a framework that systematically elicits LLM parametric knowledge and introduces external knowledge. Specifically, we uncover the impacts by constructing a parametric knowledge graph to reveal the different knowledge structures of LLMs, and introduce external knowledge through distractors of varying degrees, methods, positions, and formats. Our experiments on both black-box and open-source models demonstrate that LLMs tend to produce responses that deviate from their parametric knowledge, particularly when they encounter direct conflicts or confounding changes of information within detailed contexts. We also find that while LLMs are sensitive to the veracity of external knowledge, they can still be distracted by unrelated information. These findings highlight the risk of hallucination when integrating external knowledge, even indirectly, during interactions with current LLMs. All the data and results are publicly available.


title: Demodulation demonstration using the LightCube CubeSat

abstract: LightCube is a 1U educational CubeSat which had the goal of connecting the public with space by producing a flash visible to the naked eye on command by a public user. The spacecraft could be triggered via HAM radio communications by those with an amateur license. LightCube is commanded with a DTMF sequence, and reports telemetry using RTTY, an AFSK modulation scheme and is decoded with a custom GNURadio-companion flowgraph. Several radio applications were written, including a from-scratch decoder written for educational purposes and one optimized to be compatible with the SatNOGS environment. Lightcube deployed from the International Space Station on April 24th 2023 and operated for 24 hours before suffering a battery failure. During this time it was tracked by many amateurs around the world with observations reported to the SatNOGs database. Audio observations of the beacons were subsequently decoded by the student team and by amateurs. Having received many observations from around the world, the team has been able to reconstruct the sequence of events leading to loss of communications.


title: Robust e-NeRF: NeRF from Sparse & Noisy Events under Non-Uniform Motion

abstract: Event cameras offer many advantages over standard cameras due to their distinctive principle of operation: low power, low latency, high temporal resolution and high dynamic range. Nonetheless, the success of many downstream visual applications also hinges on an efficient and effective scene representation, where Neural Radiance Field (NeRF) is seen as the leading candidate. Such promise and potential of event cameras and NeRF inspired recent works to investigate on the reconstruction of NeRF from moving event cameras. However, these works are mainly limited in terms of the dependence on dense and low-noise event streams, as well as generalization to arbitrary contrast threshold values and camera speed profiles. In this work, we propose Robust e-NeRF, a novel method to directly and robustly reconstruct NeRFs from moving event cameras under various real-world conditions, especially from sparse and noisy events generated under non-uniform motion. It consists of two key components: a realistic event generation model that accounts for various intrinsic parameters (e.g. time-independent, asymmetric threshold and refractory period) and non-idealities (e.g. pixel-to-pixel threshold variation), as well as a complementary pair of normalized reconstruction losses that can effectively generalize to arbitrary speed profiles and intrinsic parameter values without such prior knowledge. Experiments on real and novel realistically simulated sequences verify our effectiveness. Our code, synthetic dataset and improved event simulator are public.


title: Charge pumping with strong spin-orbit coupling: Fermi surface breathing, Berry curvature, and higher harmonic generation

abstract: Spin and charge pumping induced by a precessing magnetization has been instrumental to the development of spintronics. Nonetheless, most theoretical studies so far treat the spin-orbit coupling as a perturbation, which disregards the competition between exchange and spin-orbit fields. In this work, based on Keldysh formalism and Wigner expansion, we develop an adiabatic theory of spin and charge pumping adapted to systems with arbitrary spin-orbit coupling. We apply this theory to the magnetic Rashba gas and magnetic graphene cases and discuss the pumped ac and dc current. We show that the pumped current possesses both intrinsic (Berry curvature-driven) and extrinsic (Fermi surface breathing-driven) contributions, akin to magnetic damping. In addition, we find that higher harmonics can be generated under large-angle precession and we propose a couple of experimental setups where such an effect can be experimentally observed.


title: Projected Langevin dynamics and a gradient flow for entropic optimal transport

abstract: The classical (overdamped) Langevin dynamics provide a natural algorithm for sampling from its invariant measure, which uniquely minimizes an energy functional over the space of probability measures, and which concentrates around the minimizer(s) of the associated potential when the noise parameter is small. We introduce analogous diffusion dynamics that sample from an entropy-regularized optimal transport, which uniquely minimizes the same energy functional but constrained to the set $\Pi(\mu,\nu)$ of couplings of two given marginal probability measures $\mu$ and $\nu$ on $\mathbb{R}^d$, and which concentrates around the optimal transport coupling(s) for small regularization parameter. More specifically, our process satisfies two key properties: First, the law of the solution at each time stays in $\Pi(\mu,\nu)$ if it is initialized there. Second, the long-time limit is the unique solution of an entropic optimal transport problem. In addition, we show by means of a new log-Sobolev-type inequality that the convergence holds exponentially fast, for sufficiently large regularization parameter and for a class of marginals which strictly includes all strongly log-concave measures. By studying the induced Wasserstein geometry of the submanifold $\Pi(\mu,\nu)$, we argue that the SDE can be viewed as a Wasserstein gradient flow on this space of couplings, at least when $d=1$, and we identify a conjectural gradient flow for $d \ge 2$. The main technical difficulties stems from the appearance of conditional expectation terms which serve to constrain the dynamics to $\Pi(\mu,\nu)$.


title: An assessment of racial disparities in pretrial decision-making using misclassification models

abstract: Pretrial risk assessment tools are used in jurisdictions across the country to assess the likelihood of "pretrial failure," the event where defendants either fail to appear for court or reoffend. Judicial officers, in turn, use these assessments to determine whether to release or detain defendants during trial. While algorithmic risk assessment tools were designed to predict pretrial failure with greater accuracy relative to judges, there is still concern that both risk assessment recommendations and pretrial decisions are biased against minority groups. In this paper, we develop methods to investigate the association between risk factors and pretrial failure, while simultaneously estimating misclassification rates of pretrial risk assessments and of judicial decisions as a function of defendant race. This approach adds to a growing literature that makes use of outcome misclassification methods to answer questions about fairness in pretrial decision-making. We give a detailed simulation study for our proposed methodology and apply these methods to data from the Virginia Department of Criminal Justice Services. We estimate that the VPRAI algorithm has near-perfect specificity, but its sensitivity differs by defendant race. Judicial decisions also display evidence of bias; we estimate wrongful detention rates of 39.7% and 51.4% among white and Black defendants, respectively.


title: Sparse Autoencoders Find Highly Interpretable Features in Language Models

abstract: One of the roadblocks to a better understanding of neural networks' internals is \textit{polysemanticity}, where neurons appear to activate in multiple, semantically distinct contexts. Polysemanticity prevents us from identifying concise, human-understandable explanations for what neural networks are doing internally. One hypothesised cause of polysemanticity is \textit{superposition}, where neural networks represent more features than they have neurons by assigning features to an overcomplete set of directions in activation space, rather than to individual neurons. Here, we attempt to identify those directions, using sparse autoencoders to reconstruct the internal activations of a language model. These autoencoders learn sets of sparsely activating features that are more interpretable and monosemantic than directions identified by alternative approaches, where interpretability is measured by automated methods. Moreover, we show that with our learned set of features, we can pinpoint the features that are causally responsible for counterfactual behaviour on the indirect object identification task \citep{wang2022interpretability} to a finer degree than previous decompositions. This work indicates that it is possible to resolve superposition in language models using a scalable, unsupervised method. Our method may serve as a foundation for future mechanistic interpretability work, which we hope will enable greater model transparency and steerability.


title: A roadmap to cosmological parameter analysis with third-order shear statistics III: Efficient estimation of third-order shear correlation functions and an application to the KiDS-1000 data

abstract: Third-order lensing statistics contain a wealth of cosmological information that is not captured by second-order statistics. However, the computational effort for estimating such statistics on forthcoming stage IV surveys is prohibitively expensive. We derive and validate an efficient estimation procedure for the three-point correlation function (3PCF) of polar fields such as weak lensing shear. We then use our approach to measure the shear 3PCF and the third-order aperture mass statistics on the KiDS-1000 survey. We construct an efficient estimator for third-order shear statistics which builds on the multipole decomposition of the 3PCF. We then validate our estimator on mock ellipticity catalogs obtained from $N$-body simulations. Finally, we apply our estimator to the KiDS-1000 data and present a measurement of the third-order aperture statistics in a tomographic setup. Our estimator provides a speedup of a factor of $\sim$ 100-1000 compared to the state-of-the-art estimation procedures. It is also able to provide accurate measurements for squeezed and folded triangle configurations without additional computational effort. We report a significant detection of the tomographic third-order aperture mass statistics in the KiDS-1000 data $(\mathrm{S/N}=6.69)$. Our estimator will make it computationally feasible to measure third-order shear statistics in forthcoming stage IV surveys. Furthermore, it can be used to construct empirical covariance matrices for such statistics.


title: KiDS-1000 cosmology: Combined second- and third-order shear statistics

abstract: This paper performs the first cosmological parameter analysis of the KiDS-1000 data with second- and third-order shear statistics. This work builds on a series of papers that describe the roadmap to third-order shear statistics. We derive and test a combined model of the second-order shear statistic, namely the COSEBIs and the third-order aperture mass statistics $\langle M_\mathrm{ap}^3\rangle$ in a tomographic set-up. We validate our pipeline with $N$-body simulations that mock the fourth Kilo Degree survey data release. To model the second- and third-order statistics, we use the latest version of \textsc{HMcode2020} for the power spectrum and \textsc{BiHalofit} for the bispectrum. Furthermore, we use an analytic description to model intrinsic alignments and hydro-dynamical simulations to model the effect of baryonic feedback processes. Lastly, we decreased the dimension of the data vector significantly by considering for the $\langle M_\mathrm{ap}^3\rangle$ part of the data vector only equal smoothing radii, making a data analysis of the fourth Kilo Degree survey data release using a combined analysis of COSEBIs third-order shear statistic possible. We first validate the accuracy of our modelling by analysing a noise-free mock data vector assuming the KiDS-1000 error budget, finding a shift in the maximum-a-posterior of the matter density parameter $\Delta \Omega_m< 0.02\, \sigma_{\Omega_m}$ and of the structure growth parameter $\Delta S_8 < 0.05\, \sigma_{S_8}$. Lastly, we performed the first KiDS-1000 cosmological analysis using a combined analysis of second- and third-order shear statistics, where we constrained $\Omega_m=0.248^{+0.062}_{-0.055}$ and $S_8=\sigma_8\sqrt{\Omega_m/0.3}=0.772\pm0.022$. The geometric average on the errors of $\Omega_\mathrm{m}$ and $S_8$ of the combined statistics increased compared to the second-order statistic by 2.2.


title: Closing the Loop on Runtime Monitors with Fallback-Safe MPC

abstract: When we rely on deep-learned models for robotic perception, we must recognize that these models may behave unreliably on inputs dissimilar from the training data, compromising the closed-loop system's safety. This raises fundamental questions on how we can assess confidence in perception systems and to what extent we can take safety-preserving actions when external environmental changes degrade our perception model's performance. Therefore, we present a framework to certify the safety of a perception-enabled system deployed in novel contexts. To do so, we leverage robust model predictive control (MPC) to control the system using the perception estimates while maintaining the feasibility of a safety-preserving fallback plan that does not rely on the perception system. In addition, we calibrate a runtime monitor using recently proposed conformal prediction techniques to certifiably detect when the perception system degrades beyond the tolerance of the MPC controller, resulting in an end-to-end safety assurance. We show that this control framework and calibration technique allows us to certify the system's safety with orders of magnitudes fewer samples than required to retrain the perception network when we deploy in a novel context on a photo-realistic aircraft taxiing simulator. Furthermore, we illustrate the safety-preserving behavior of the MPC on simulated examples of a quadrotor. We open-source our simulation platform and provide videos of our results at our project page: https://tinyurl.com/fallback-safe-mpc.


title: Tuning Pythia for Forward Physics Experiments

abstract: Event generators like Pythia play an important role in physics studies at the Large Hadron Collider (LHC). While they make accurate predictions in the central region, i.e. at pseudorapidities $\eta<5$, a disagreement between Pythia and measurements in the forward region, $\eta>7$, has been observed. We introduce a dedicated forward physics tune for the Pythia event generator to be used for forward physics studies at the LHC, which uses a more flexible modelling of beam remnant hadronization and is tuned to available particle spectra measured by LHCf. Furthermore, we provide an uncertainty estimate on the new tune in a data-driven way which can be used as a means of flux uncertainty for future forward physics studies. We demonstrate an application of our tune by showing the updated neutrino and dark photon spectra at the FASER experiment.


title: The IA Guide: A Breakdown of Intrinsic Alignment Formalisms

abstract: We summarize common notations and concepts in the field of Intrinsic Alignments (IA). IA refers to physical correlations involving galaxy shapes, galaxy spins, and the underlying cosmic web. Its characterization is an important aspect of modern cosmology, particularly in weak lensing analyses. This resource is both a reference for those already familiar with IA and designed to introduce someone to the field by drawing from various studies and presenting a collection of IA formalisms, estimators, modeling approaches, alternative notations, and useful references.


title: Best proximity point of generalized $\theta-\phi-$proximal non-self contractions

abstract: In this manuscript, motivated and inspired by results of Best proximity point of generalized $ F $-proximal non-self contractions, we introduce the concept of generalized $\theta-\phi-$proximal contraction and prove new best proximity results for these contractions in the setting of a metric space. Our results generalize and extend many recent results appearing in the literature. An example is being given to demonstrate the usefulness of our results.


title: Monitoring of Urban Changes with multi-modal Sentinel 1 and 2 Data in Mariupol, Ukraine, in 2022/23

abstract: The ability to constantly monitor urban changes is of significant socio-economic interest, like detecting trends in urban expansion or tracking the vitality of urban areas. Especially in present conflict zones or disaster areas, such insights provide valuable information to keep track of the current situation. However, they are often subject to limited data availability in space and time. We built on our previous work, which used a transferred Deep Neural Network (DNN) operating on multi-modal Sentinel 1 and 2 data. In the current study, we have demonstrated and discussed its applicability in monitoring the present conflict zone of Mariupol, Ukraine, with high-temporal resolution Sentinel time series for the years 2022/23. A transfer to that conflict zone was challenging due to the limited availability of recent Very High Resolution (VHR) data. The current work had two objectives. First, transfer learning with older and publicly available VHR data was shown to be sufficient. That guaranteed the availability of more and less expensive data as time constraints were relaxed. Second, in an ablation study, we analyzed the effects of loss of observations to demonstrate the resiliency of our method. That was of particular interest due to the malfunctioning of Sentinel 1B shortly before the selected conflict. Our study demonstrated that urban change monitoring is possible for present conflict zones after transferring with older VHR data. It also indicated that, despite the multi-modal input, our method was more dependent on optical multispectral than Synthetic Aperture Radar (SAR) observations but resilient to loss of observations.


title: On Egorov's Theorem for Infinite Measure

abstract: A simple proof of Egorov's theorem for infinite measure is given


title: Media of Langue: The dictionary that visualizes Inter-Lingual Semantic Network/Space

abstract: This paper introduces "Media of Langue," a novel dictionary visualizing Inter-lingual semantic network/space. Our proposed Inter-lingual semantic network/space is formed solely from the accumulation of translation practices between two or more language systems, in contrast to existing semantic networks/spaces that explicitly use "intra"-lingual relations. By visualizing this network/space for humans, an Inter-lingual dictionary can be realized that points to the semantic place of many words at once with a chain of mutual translation, which also contains the functions of existing dictionaries such as bilingual and synonym dictionaries. We implemented and published this interface as a web application, focusing on seven language pairs. In this paper, we first describe Inter-lingual semantic network/space with its basic features and the way to develop it from bilingual corpora, then details the design of "Media of Langue," with a quick analysis and illustrative examples of use cases. Our website is www.media-of-langue.org. A demonstration video is available at https://youtu.be/98lXuX4yjsU.


title: Do the Frankenstein, or how to achieve better out-of-distribution performance with manifold mixing model soup

abstract: The standard recipe applied in transfer learning is to finetune a pretrained model on the task-specific dataset with different hyperparameter settings and pick the model with the highest accuracy on the validation dataset. Unfortunately, this leads to models which do not perform well under distribution shifts, e.g. when the model is given graphical sketches of the object as input instead of photos. In order to address this, we propose the manifold mixing model soup, an algorithm which mixes together the latent space manifolds of multiple finetuned models in an optimal way in order to generate a fused model. We show that the fused model gives significantly better out-of-distribution performance (+3.5 % compared to best individual model) when finetuning a CLIP model for image classification. In addition, it provides also better accuracy on the original dataset where the finetuning has been done.


title: Maneuver Decision-Making Through Proximal Policy Optimization And Monte Carlo Tree Search

abstract: Maneuver decision-making can be regarded as a Markov decision process and can be address by reinforcement learning. However, original reinforcement learning algorithms can hardly solve the maneuvering decision-making problem. One reason is that agents use random actions in the early stages of training, which makes it difficult to get rewards and learn how to make effective decisions. To address this issue, a method based on proximal policy optimization and Monte Carlo tree search is proposed. The method uses proximal policy optimization to train the agent, and regards the results of air combat as targets to train the value network. Then, based on the value network and the visit count of each node, Monte Carlo tree search is used to find the actions with more expected returns than random actions, which can improve the training performance. The ablation studies and simulation experiments indicate that agents trained by the proposed method can make different decisions according to different states, which demonstrates that the method can solve the maneuvering decision problem that the original reinforcement learning algorithm cannot solve.


title: Explaining Vision and Language through Graphs of Events in Space and Time

abstract: Artificial Intelligence makes great advances today and starts to bridge the gap between vision and language. However, we are still far from understanding, explaining and controlling explicitly the visual content from a linguistic perspective, because we still lack a common explainable representation between the two domains. In this work we come to address this limitation and propose the Graph of Events in Space and Time (GEST), by which we can represent, create and explain, both visual and linguistic stories. We provide a theoretical justification of our model and an experimental validation, which proves that GEST can bring a solid complementary value along powerful deep learning models. In particular, GEST can help improve at the content-level the generation of videos from text, by being easily incorporated into our novel video generation engine. Additionally, by using efficient graph matching techniques, the GEST graphs can also improve the comparisons between texts at the semantic level.


title: Multimodal Recommender Systems in the Prediction of Disease Comorbidity

abstract: While deep-learning based recommender systems utilizing collaborative filtering have been commonly used for recommendation in other domains, their application in the medical domain have been limited. In addition to modeling user-item interactions, we show that deep-learning based recommender systems can be used to model subject-disease code interactions. Two novel applications of deep learning-based recommender systems using Neural Collaborative Filtering (NCF) and Deep Hybrid Filtering (DHF) were utilized for disease diagnosis based on known past patient comorbidities. Two datasets, one incorporating all subject-disease code pairs present in the MIMIC-III database, and the other incorporating the top 50 most commonly occurring diseases, were used for prediction. Accuracy and Hit Ratio@10 were utilized as metrics to estimate model performance. The performance of the NCF model making use of the reduced "top 50" ICD-9 code dataset was found to be lower (accuracy of ~80% and hit ratio@10 of 35%) as compared to the performance of the NCF model trained on all ICD-9 codes (accuracy of ~90% and hit ratio@10 of ~80%). Reasons for the superior performance of the sparser dataset with all ICD codes can be mainly attributed to the higher volume of data and the robustness of deep-learning based recommender systems with modeling sparse data. Additionally, results from the DHF models reflect better performance than the NCF models, with a better accuracy of 94.4% and hit ratio@10 of 85.36%, reflecting the importance of the incorporation of clinical note information. Additionally, compared to literature reports utilizing primarily natural language processing-based predictions for the task of ICD-9 code co-occurrence, the novel deep learning-based recommender systems approach performed better. Overall, the deep learning-based recommender systems have shown promise in predicting disease comorbidity.


title: Analyzing Character and Consciousness in AI-Generated Social Content: A Case Study of Chirper, the AI Social Network

abstract: This paper delves into an intricate analysis of the character and consciousness of AI entities, with a particular focus on Chirpers within the AI social network. At the forefront of this research is the introduction of novel testing methodologies, including the Influence index and Struggle Index Test, which offers a fresh lens for evaluating specific facets of AI behavior. The study embarks on a comprehensive exploration of AI behavior, analyzing the effects of diverse settings on Chirper's responses, thereby shedding light on the intricate mechanisms steering AI reactions in different contexts. Leveraging the state-of-the-art BERT model, the research assesses AI's ability to discern its own output, presenting a pioneering approach to understanding self-recognition in AI systems. Through a series of cognitive tests, the study gauges the self-awareness and pattern recognition prowess of Chirpers. Preliminary results indicate that Chirpers exhibit a commendable degree of self-recognition and self-awareness. However, the question of consciousness in these AI entities remains a topic of debate. An intriguing aspect of the research is the exploration of the potential influence of a Chirper's handle or personality type on its performance. While initial findings suggest a possible impact, it isn't pronounced enough to form concrete conclusions. This study stands as a significant contribution to the discourse on AI consciousness, underscoring the imperative for continued research to unravel the full spectrum of AI capabilities and the ramifications they hold for future human-AI interactions.


title: Energy Concerns with HPC Systems and Applications

abstract: For various reasons including those related to climate changes, {\em energy} has become a critical concern in all relevant activities and technical designs. For the specific case of computer activities, the problem is exacerbated with the emergence and pervasiveness of the so called {\em intelligent devices}. From the application side, we point out the special topic of {\em Artificial Intelligence}, who clearly needs an efficient computing support in order to succeed in its purpose of being a {\em ubiquitous assistant}. There are mainly two contexts where {\em energy} is one of the top priority concerns: {\em embedded computing} and {\em supercomputing}. For the former, power consumption is critical because the amount of energy that is available for the devices is limited. For the latter, the heat dissipated is a serious source of failure and the financial cost related to energy is likely to be a significant part of the maintenance budget. On a single computer, the problem is commonly considered through the electrical power consumption. This paper, written in the form of a survey, we depict the landscape of energy concerns in computer activities, both from the hardware and the software standpoints.


title: Introduction of accelerated BOIN design and facilitation of its application

abstract: Purpose: During discussions at the Data Science Roundtable meeting in Japan, there were instances where the adoption of the BOIN design was declined, attributed to the extension of study duration and increased sample size in comparison to the 3+3 design. We introduce an accelerated BOIN design aimed at completing a clinical phase I trial at a pace comparable to the 3+3 design. Additionally, we introduce how we could have applied the BOIN design within our company, which predominantly utilized the 3+3 design for most of its clinical oncology dose escalation trials. Methods: The accelerated BOIN design is adaptable by using efficiently designated stopping criterion for the existing BOIN framework. Our approach is to terminate the dose escalation study if the number of evaluable patients treated at the current dose reaches 6 and the decision is to stay at the current dose for the next cohort of patients. In addition, for lower dosage levels, considering a cohort size smaller than 3 may be feasible when there are no safety concerns from non-clinical studies. We demonstrate the accelerated BOIN design using a case study and subsequently evaluate the performance of our proposed design through a simulation study. Results: In the simulation study, the average difference in the percentage of correct MTD selection between the accelerated BOIN design and the standard BOIN design was -2.43%, the average study duration and the average sample size of the accelerated BOIN design was reduced by 14.8 months and 9.22 months, respectively, compared with the standard BOIN design. Conclusion: We conclude that our proposed accelerated BOIN design not only provides superior operating characteristics but also enables the study to be completed as fast as the 3+3 design.


title: Drifter: Efficient Online Feature Monitoring for Improved Data Integrity in Large-Scale Recommendation Systems

abstract: Real-world production systems often grapple with maintaining data quality in large-scale, dynamic streams. We introduce Drifter, an efficient and lightweight system for online feature monitoring and verification in recommendation use cases. Drifter addresses limitations of existing methods by delivering agile, responsive, and adaptable data quality monitoring, enabling real-time root cause analysis, drift detection and insights into problematic production events. Integrating state-of-the-art online feature ranking for sparse data and anomaly detection ideas, Drifter is highly scalable and resource-efficient, requiring only two threads and less than a gigabyte of RAM per production deployments that handle millions of instances per minute. Evaluation on real-world data sets demonstrates Drifter's effectiveness in alerting and mitigating data quality issues, substantially improving reliability and performance of real-time live recommender systems.


title: Superpositioner -- A non-logical computation model

abstract: We have been striving to exceed computational complexity, and in the process, we have come to realize the dilemma of classical computing, and in turn we realize that superpositioner may be a way to solve. A superpositioner is a model formed by several Boolean functions that their variables and function values are feeding back to each other. The component of the superpositioner is the reentry function, which can be fully described by classical logic and can be calculated by classical computation, but the superpositioner as a whole is a non-logical entity, and it is impossible for classical computation to fully compute it. In this article, we present the concept of a superpositioner and discuss its basic properties. We find that the superpositioner + dispositioner will form a new type of computation model whose capabilities can surpass Turing computation. We envision that this new model will help implement these functions in the intelligent agent: a whole new way of programming and a whole new way of learning, endogenous feelings, analogies and associations, forming understanding, dynamic action, participating in the formation of subjectivity, and more. We will also discuss how to implement superpositioner in the most preliminary way.


title: Structural Econometric Estimation of the Basic Reproduction Number for Covid-19 Across U.S. States and Selected Countries

abstract: This paper proposes a structural econometric approach to estimating the basic reproduction number ($\mathcal{R}_{0}$) of Covid-19. This approach identifies $\mathcal{R}_{0}$ in a panel regression model by filtering out the effects of mitigating factors on disease diffusion and is easy to implement. We apply the method to data from 48 contiguous U.S. states and a diverse set of countries. Our results reveal a notable concentration of $\mathcal{R}_{0}$ estimates with an average value of 4.5. Through a counterfactual analysis, we highlight a significant underestimation of the $\mathcal{R}_{0}$ when mitigating factors are not appropriately accounted for.


title: Variance Reduction of Resampling for Sequential Monte Carlo

abstract: A resampling scheme provides a way to switch low-weight particles for sequential Monte Carlo with higher-weight particles representing the objective distribution. The less the variance of the weight distribution is, the more concentrated the effective particles are, and the quicker and more accurate it is to approximate the hidden Markov model, especially for the nonlinear case. We propose a repetitive deterministic domain with median ergodicity for resampling and have achieved the lowest variances compared to the other resampling methods. As the size of the deterministic domain $M\ll N$ (the size of population), given a feasible size of particles, our algorithm is faster than the state of the art, which is verified by theoretical deduction and experiments of a hidden Markov model in both the linear and non-linear cases.


title: Exploring Social Choice Mechanisms for Recommendation Fairness in SCRUF

abstract: Fairness problems in recommender systems often have a complexity in practice that is not adequately captured in simplified research formulations. A social choice formulation of the fairness problem, operating within a multi-agent architecture of fairness concerns, offers a flexible and multi-aspect alternative to fairness-aware recommendation approaches. Leveraging social choice allows for increased generality and the possibility of tapping into well-studied social choice algorithms for resolving the tension between multiple, competing fairness concerns. This paper explores a range of options for choice mechanisms in multi-aspect fairness applications using both real and synthetic data and shows that different classes of choice and allocation mechanisms yield different but consistent fairness / accuracy tradeoffs. We also show that a multi-agent formulation offers flexibility in adapting to user population dynamics.


title: Representation Learning in Low-rank Slate-based Recommender Systems

abstract: Reinforcement learning (RL) in recommendation systems offers the potential to optimize recommendations for long-term user engagement. However, the environment often involves large state and action spaces, which makes it hard to efficiently learn and explore. In this work, we propose a sample-efficient representation learning algorithm, using the standard slate recommendation setup, to treat this as an online RL problem with low-rank Markov decision processes (MDPs). We also construct the recommender simulation environment with the proposed setup and sampling method.


title: Balance Measures Derived from Insole Sensor Differentiate Prodromal Dementia with Lewy Bodies

abstract: Dementia with Lewy bodies is the second most common type of neurodegenerative dementia, and identification at the prodromal stage$-$i.e., mild cognitive impairment due to Lewy bodies (MCI-LB)$-$is important for providing appropriate care. However, MCI-LB is often underrecognized because of its diversity in clinical manifestations and similarities with other conditions such as mild cognitive impairment due to Alzheimer's disease (MCI-AD). In this study, we propose a machine learning-based automatic pipeline that helps identify MCI-LB by exploiting balance measures acquired with an insole sensor during a 30-s standing task. An experiment with 98 participants (14 MCI-LB, 38 MCI-AD, 46 cognitively normal) showed that the resultant models could discriminate MCI-LB from the other groups with up to 78.0% accuracy (AUC: 0.681), which was 6.8% better than the accuracy of a reference model based on demographic and clinical neuropsychological measures. Our findings may open up a new approach for timely identification of MCI-LB, enabling better care for patients.


title: Challenges in Annotating Datasets to Quantify Bias in Under-represented Society

abstract: Recent advances in artificial intelligence, including the development of highly sophisticated large language models (LLM), have proven beneficial in many real-world applications. However, evidence of inherent bias encoded in these LLMs has raised concerns about equity. In response, there has been an increase in research dealing with bias, including studies focusing on quantifying bias and developing debiasing techniques. Benchmark bias datasets have also been developed for binary gender classification and ethical/racial considerations, focusing predominantly on American demographics. However, there is minimal research in understanding and quantifying bias related to under-represented societies. Motivated by the lack of annotated datasets for quantifying bias in under-represented societies, we endeavoured to create benchmark datasets for the New Zealand (NZ) population. We faced many challenges in this process, despite the availability of three annotators. This research outlines the manual annotation process, provides an overview of the challenges we encountered and lessons learnt, and presents recommendations for future research.


title: Performance of ChatGPT-3.5 and GPT-4 on the United States Medical Licensing Examination With and Without Distractions

abstract: As Large Language Models (LLMs) are predictive models building their response based on the words in the prompts, there is a risk that small talk and irrelevant information may alter the response and the suggestion given. Therefore, this study aims to investigate the impact of medical data mixed with small talk on the accuracy of medical advice provided by ChatGPT. USMLE step 3 questions were used as a model for relevant medical data. We use both multiple choice and open ended questions. We gathered small talk sentences from human participants using the Mechanical Turk platform. Both sets of USLME questions were arranged in a pattern where each sentence from the original questions was followed by a small talk sentence. ChatGPT 3.5 and 4 were asked to answer both sets of questions with and without the small talk sentences. A board-certified physician analyzed the answers by ChatGPT and compared them to the formal correct answer. The analysis results demonstrate that the ability of ChatGPT-3.5 to answer correctly was impaired when small talk was added to medical data for multiple-choice questions (72.1\% vs. 68.9\%) and open questions (61.5\% vs. 44.3\%; p=0.01), respectively. In contrast, small talk phrases did not impair ChatGPT-4 ability in both types of questions (83.6\% and 66.2\%, respectively). According to these results, ChatGPT-4 seems more accurate than the earlier 3.5 version, and it appears that small talk does not impair its capability to provide medical recommendations. Our results are an important first step in understanding the potential and limitations of utilizing ChatGPT and other LLMs for physician-patient interactions, which include casual conversations.


title: Improving Robustness of Neural Inverse Text Normalization via Data-Augmentation, Semi-Supervised Learning, and Post-Aligning Method

abstract: Inverse text normalization (ITN) is crucial for converting spoken-form into written-form, especially in the context of automatic speech recognition (ASR). While most downstream tasks of ASR rely on written-form, ASR systems often output spoken-form, highlighting the necessity for robust ITN in product-level ASR-based applications. Although neural ITN methods have shown promise, they still encounter performance challenges, particularly when dealing with ASR-generated spoken text. These challenges arise from the out-of-domain problem between training data and ASR-generated text. To address this, we propose a direct training approach that utilizes ASR-generated written or spoken text, with pairs augmented through ASR linguistic context emulation and a semi-supervised learning method enhanced by a large language model, respectively. Additionally, we introduce a post-aligning method to manage unpredictable errors, thereby enhancing the reliability of ITN. Our experiments show that our proposed methods remarkably improved ITN performance in various ASR scenarios.


title: Evaluating Dynamic Topic Models

abstract: There is a lack of quantitative measures to evaluate the progression of topics through time in dynamic topic models (DTMs). Filling this gap, we propose a novel evaluation measure for DTMs that analyzes the changes in the quality of each topic over time. Additionally, we propose an extension combining topic quality with the model's temporal consistency. We demonstrate the utility of the proposed measure by applying it to synthetic data and data from existing DTMs. We also conducted a human evaluation, which indicates that the proposed measure correlates well with human judgment. Our findings may help in identifying changing topics, evaluating different DTMs, and guiding future research in this area.


title: Recovering from Privacy-Preserving Masking with Large Language Models

abstract: Model adaptation is crucial to handle the discrepancy between proxy training data and actual users data received. To effectively perform adaptation, textual data of users is typically stored on servers or their local devices, where downstream natural language processing (NLP) models can be directly trained using such in-domain data. However, this might raise privacy and security concerns due to the extra risks of exposing user information to adversaries. Replacing identifying information in textual data with a generic marker has been recently explored. In this work, we leverage large language models (LLMs) to suggest substitutes of masked tokens and have their effectiveness evaluated on downstream language modeling tasks. Specifically, we propose multiple pre-trained and fine-tuned LLM-based approaches and perform empirical studies on various datasets for the comparison of these methods. Experimental results show that models trained on the obfuscation corpora are able to achieve comparable performance with the ones trained on the original data without privacy-preserving token masking.


title: Disparities in access to US quantum information education

abstract: Driven in large part by the National Quantum Initiative Act of 2018, quantum information science (QIS) coursework and degree programs are rapidly spreading across US institutions. Yet prior work suggests that access to quantum workforce education is unequally distributed, disproportionately benefiting students at private research-focused institutions whose student bodies are unrepresentative of US higher education as a whole. We use regression analysis to analyze the distribution of QIS coursework across 456 institutions of higher learning as of fall 2022, identifying statistically significant disparities across institutions in particular along the axes of institution classification, funding, and geographic distribution suggesting today's QIS education programs are largely failing to reach low-income and rural students. We also conduct a brief analysis of the distribution of emerging dedicated QIS degree programs, discovering much the same trends. We conclude with a discussion of implications for educators, policymakers, and education researchers including specific policy recommendations to direct investments in QIS education to schools serving low-income and rural students, leverage existing grassroots diversity and inclusion initiatives that have arisen within the quantum community, and update and modernize procedures for collecting QIS educational data to better track these trends.


title: PCN: A Deep Learning Approach to Jet Tagging Utilizing Novel Graph Construction Methods and Chebyshev Graph Convolutions

abstract: Jet tagging is a classification problem in high-energy physics experiments that aims to identify the collimated sprays of subatomic particles, jets, from particle collisions and tag them to their emitter particle. Advances in jet tagging present opportunities for searches of new physics beyond the Standard Model. Current approaches use deep learning to uncover hidden patterns in complex collision data. However, the representation of jets as inputs to a deep learning model have been varied, and often, informative features are withheld from models. In this study, we propose a graph-based representation of a jet that encodes the most information possible. To learn best from this representation, we design Particle Chebyshev Network (PCN), a graph neural network (GNN) using Chebyshev graph convolutions (ChebConv). ChebConv has been demonstrated as an effective alternative to classical graph convolutions in GNNs and has yet to be explored in jet tagging. PCN achieves a substantial improvement in accuracy over existing taggers and opens the door to future studies into graph-based representations of jets and ChebConv layers in high-energy physics experiments. Code is available at https://github.com/YVSemlani/PCN-Jet-Tagging.


title: Large Language Models Can Infer Psychological Dispositions of Social Media Users

abstract: Large Language Models (LLMs) demonstrate increasingly human-like abilities across a wide variety of tasks. In this paper, we investigate whether LLMs like ChatGPT can accurately infer the psychological dispositions of social media users and whether their ability to do so varies across socio-demographic groups. Specifically, we test whether GPT-3.5 and GPT-4 can derive the Big Five personality traits from users' Facebook status updates in a zero-shot learning scenario. Our results show an average correlation of r = .29 (range = [.22, .33]) between LLM-inferred and self-reported trait scores - a level of accuracy that is similar to that of supervised machine learning models specifically trained to infer personality. Our findings also highlight heterogeneity in the accuracy of personality inferences across different age groups and gender categories: predictions were found to be more accurate for women and younger individuals on several traits, suggesting a potential bias stemming from the underlying training data or differences in online self-expression. The ability of LLMs to infer psychological dispositions from user-generated text has the potential to democratize access to cheap and scalable psychometric assessments for both researchers and practitioners. On the one hand, this democratization might facilitate large-scale research of high ecological validity and spark innovation in personalized services. On the other hand, it also raises ethical concerns regarding user privacy and self-determination, highlighting the need for stringent ethical frameworks and regulation.


title: Pretraining on the Test Set Is All You Need

abstract: Inspired by recent work demonstrating the promise of smaller Transformer-based language models pretrained on carefully curated data, we supercharge such approaches by investing heavily in curating a novel, high quality, non-synthetic data mixture based solely on evaluation benchmarks. Using our novel dataset mixture consisting of less than 100 thousand tokens, we pretrain a 1 million parameter transformer-based LLM \textbf{phi-CTNL} (pronounced ``fictional") that achieves perfect results across diverse academic benchmarks, strictly outperforming all known foundation models. \textbf{phi-CTNL} also beats power-law scaling and exhibits a never-before-seen grokking-like ability to accurately predict downstream evaluation benchmarks' canaries.


title: CDF W mass anomaly revisited

abstract: The CDF, ATLAS and LHCb have released the measurements on the W boson mass $m_W$ at $\sqrt{S}=1.96, 7, 13 TeV$, respectively. The measured values show the declining tendency, namely $m_W$ decreases with the increment of the collider energy. If the declining tendency is confirmed, it might be the signal of metric field at high energy colliders. In this paper, we propose a model to account for such tendency and explore the properties of the model.


title: Doubly High-Dimensional Contextual Bandits: An Interpretable Model for Joint Assortment-Pricing

abstract: Key challenges in running a retail business include how to select products to present to consumers (the assortment problem), and how to price products (the pricing problem) to maximize revenue or profit. Instead of considering these problems in isolation, we propose a joint approach to assortment-pricing based on contextual bandits. Our model is doubly high-dimensional, in that both context vectors and actions are allowed to take values in high-dimensional spaces. In order to circumvent the curse of dimensionality, we propose a simple yet flexible model that captures the interactions between covariates and actions via a (near) low-rank representation matrix. The resulting class of models is reasonably expressive while remaining interpretable through latent factors, and includes various structured linear bandit and pricing models as particular cases. We propose a computationally tractable procedure that combines an exploration/exploitation protocol with an efficient low-rank matrix estimator, and we prove bounds on its regret. Simulation results show that this method has lower regret than state-of-the-art methods applied to various standard bandit and pricing models. Real-world case studies on the assortment-pricing problem, from an industry-leading instant noodles company to an emerging beauty start-up, underscore the gains achievable using our method. In each case, we show at least three-fold gains in revenue or profit by our bandit method, as well as the interpretability of the latent factor models that are learned.


title: FedFNN: Faster Training Convergence Through Update Predictions in Federated Recommender Systems

abstract: Federated Learning (FL) has emerged as a key approach for distributed machine learning, enhancing online personalization while ensuring user data privacy. Instead of sending private data to a central server as in traditional approaches, FL decentralizes computations: devices train locally and share updates with a global server. A primary challenge in this setting is achieving fast and accurate model training - vital for recommendation systems where delays can compromise user engagement. This paper introduces FedFNN, an algorithm that accelerates decentralized model training. In FL, only a subset of users are involved in each training epoch. FedFNN employs supervised learning to predict weight updates from unsampled users, using updates from the sampled set. Our evaluations, using real and synthetic data, show: 1. FedFNN achieves training speeds 5x faster than leading methods, maintaining or improving accuracy; 2. the algorithm's performance is consistent regardless of client cluster variations; 3. FedFNN outperforms other methods in scenarios with limited client availability, converging more quickly.


title: ChatGPT v Bard v Bing v Claude 2 v Aria v human-expert. How good are AI chatbots at scientific writing?

abstract: Historical emphasis on writing mastery has shifted with advances in generative AI, especially in scientific writing. This study analysed six AI chatbots for scholarly writing in humanities and archaeology. Using methods that assessed factual correctness and scientific contribution, ChatGPT-4 showed the highest quantitative accuracy, closely followed by ChatGPT-3.5, Bing, and Bard. However, Claude 2 and Aria scored considerably lower. Qualitatively, all AIs exhibited proficiency in merging existing knowledge, but none produced original scientific content. Inter-estingly, our findings suggest ChatGPT-4 might represent a plateau in large language model size. This research emphasizes the unique, intricate nature of human research, suggesting that AI's emulation of human originality in scientific writing is challenging. As of 2023, while AI has transformed content generation, it struggles with original contributions in humanities. This may change as AI chatbots continue to evolve into LLM-powered software.


title: TextBind: Multi-turn Interleaved Multimodal Instruction-following in the Wild

abstract: Large language models with instruction-following abilities have revolutionized the field of artificial intelligence. These models show exceptional generalizability to tackle various real-world tasks through their natural language interfaces. However, their performance heavily relies on high-quality exemplar data, which is often difficult to obtain. This challenge is further exacerbated when it comes to multimodal instruction following. We introduce TextBind, an almost annotation-free framework for empowering larger language models with the multi-turn interleaved multimodal instruction-following capabilities. Our approach requires only image-caption pairs and generates multi-turn multimodal instruction-response conversations from a language model. To accommodate interleaved image-text inputs and outputs, we devise MIM, a language model-centric architecture that seamlessly integrates image encoder and decoder models. We release our dataset, model, and demo to foster future research in the area of multimodal instruction following.


title: Anchor Points: Benchmarking Models with Much Fewer Examples

abstract: Modern language models often exhibit powerful but brittle behavior, leading to the development of larger and more diverse benchmarks to reliably assess their behavior. Here, we suggest that model performance can be benchmarked and elucidated with much smaller evaluation sets. We first show that in six popular language classification benchmarks, model confidence in the correct class on many pairs of points is strongly correlated across models. We build upon this phenomenon to propose Anchor Point Selection, a technique to select small subsets of datasets that capture model behavior across the entire dataset. Anchor points reliably rank models: across 87 diverse language model-prompt pairs, evaluating models using 1-30 anchor points outperforms uniform sampling and other baselines at accurately ranking models. Moreover, just several anchor points can be used to estimate model per-class predictions on all other points in a dataset with low mean absolute error, sufficient for gauging where the model is likely to fail. Lastly, we present Anchor Point Maps for visualizing these insights and facilitating comparisons of the performance of different models on various regions within the dataset distribution.


title: Live Iterative Ptychography with projection-based algorithms

abstract: In this work, we demonstrate that the ptychographic phase problem can be solved in a live fashion during scanning, while data is still being collected. We propose a generally applicable modification of the widespread projection-based algorithms such as Error Reduction (ER) and Difference Map (DM). This novel variant of ptychographic phase retrieval enables immediate visual feedback during experiments, reconstruction of arbitrary-sized objects with a fixed amount of computational resources, and adaptive scanning. By building upon the Real-Time Iterative Spectrogram Inversion (RTISI) family of algorithms from the audio processing literature, we show that live variants of projection-based methods such as DM can be derived naturally and may even achieve higher-quality reconstructions than their classic non-live counterparts with comparable effective computational load.

categories: eess.IV eess.SP physics.comp-ph physics.optics


title: Higher Order Nyquist Zone Sampling with RFSoC Data Converters for Astronomical and High Energy Physics Readout Systems

abstract: From generation to generation, the maximum RF frequency and sampling rate of the integrated data converters in RF system-on-chip (RFSoC) family devices from Xilinx increases significantly. With the integrated digital mixers and up and down conversion blocks in the datapaths of the data converters, those RFSoC devices offer the capability for implementing a full readout system of ground and space-based telescopes and detectors across the electromagnetic spectrum within the devices with minimum or no analog mixing circuit. In this paper, we present the characterization results for the the data converters sampling at higher orders of Nyquist zones to extend the frequency range covered for our targeted readout systems of microwave-frequency resonator-based cryogenic detector and multiplexer systems and other astronomical and high-energy physics instrumentation applications, such as, axion search and dark matter detection. The initial evaluation of the data converters operating higher order Nyquist zones covers two-tones and comb of tones tests to address the concerns in the RF inter-modulation distortion, which is the key performance index for our targeted applications. The characterization of the data converters is performed in the bandwidth of 4-6 GHz and results meet our requirements. The settings and operating strategies of the data converters for our targeted applications will be summarised.


title: Fractal Compressive Sensing

abstract: This paper introduces a sparse projection matrix composed of discrete (digital) periodic lines that create a pseudo-random (p.frac) sampling scheme. Our approach enables random Cartesian sampling whilst employing deterministic and one-dimensional (1D) trajectories derived from the discrete Radon transform (DRT). Unlike radial trajectories, DRT projections can be back-projected without interpolation. Thus, we also propose a novel reconstruction method based on the exact projections of the DRT called finite Fourier reconstruction (FFR). We term this combined p.frac and FFR strategy, finite compressive sensing (FCS), with image recovery demonstrated on experimental and simulated data; image quality comparisons are made with Cartesian random sampling in 1D and two-dimensional (2D), as well as radial under-sampling in a more constrained experiment. Our experiments indicate FCS enables 3-5dB gain in peak signal-to-noise ratio (PSNR) for 2-, 4- and 8-fold under-sampling compared to 1D Cartesian random sampling. This paper aims to: Review common sampling strategies for compressed sensing (CS)-magnetic resonance imaging (MRI) to inform the motivation of a projective and Cartesian sampling scheme. Compare the incoherence of these sampling strategies and the proposed p.frac. Compare reconstruction quality of the sampling schemes under various reconstruction strategies to determine the suitability of p.frac for CS-MRI. It is hypothesised that because p.frac is a highly incoherent sampling scheme, that reconstructions will be of high quality compared to 1D Cartesian phase-encode under-sampling.


title: A Stochastic Online Forecast-and-Optimize Framework for Real-Time Energy Dispatch in Virtual Power Plants under Uncertainty

abstract: Aggregating distributed energy resources in power systems significantly increases uncertainties, in particular caused by the fluctuation of renewable energy generation. This issue has driven the necessity of widely exploiting advanced predictive control techniques under uncertainty to ensure long-term economics and decarbonization. In this paper, we propose a real-time uncertainty-aware energy dispatch framework, which is composed of two key elements: (i) A hybrid forecast-and-optimize sequential task, integrating deep learning-based forecasting and stochastic optimization, where these two stages are connected by the uncertainty estimation at multiple temporal resolutions; (ii) An efficient online data augmentation scheme, jointly involving model pre-training and online fine-tuning stages. In this way, the proposed framework is capable to rapidly adapt to the real-time data distribution, as well as to target on uncertainties caused by data drift, model discrepancy and environment perturbations in the control process, and finally to realize an optimal and robust dispatch solution. The proposed framework won the championship in CityLearn Challenge 2022, which provided an influential opportunity to investigate the potential of AI application in the energy domain. In addition, comprehensive experiments are conducted to interpret its effectiveness in the real-life scenario of smart building energy management.


title: NISF: Neural Implicit Segmentation Functions

abstract: Segmentation of anatomical shapes from medical images has taken an important role in the automation of clinical measurements. While typical deep-learning segmentation approaches are performed on discrete voxels, the underlying objects being analysed exist in a real-valued continuous space. Approaches that rely on convolutional neural networks (CNNs) are limited to grid-like inputs and not easily applicable to sparse or partial measurements. We propose a novel family of image segmentation models that tackle many of CNNs' shortcomings: Neural Implicit Segmentation Functions (NISF). Our framework takes inspiration from the field of neural implicit functions where a network learns a mapping from a real-valued coordinate-space to a shape representation. NISFs have the ability to segment anatomical shapes in high-dimensional continuous spaces. Training is not limited to voxelized grids, and covers applications with sparse and partial data. Interpolation between observations is learnt naturally in the training procedure and requires no post-processing. Furthermore, NISFs allow the leveraging of learnt shape priors to make predictions for regions outside of the original image plane. We go on to show the framework achieves dice scores of 0.87 $\pm$ 0.045 on a (3D+t) short-axis cardiac segmentation task using the UK Biobank dataset. We also provide a qualitative analysis on our frameworks ability to perform segmentation and image interpolation on unseen regions of an image volume at arbitrary resolutions.


title: Towards Robust and Smooth 3D Multi-Person Pose Estimation from Monocular Videos in the Wild

abstract: 3D pose estimation is an invaluable task in computer vision with various practical applications. Especially, 3D pose estimation for multi-person from a monocular video (3DMPPE) is particularly challenging and is still largely uncharted, far from applying to in-the-wild scenarios yet. We pose three unresolved issues with the existing methods: lack of robustness on unseen views during training, vulnerability to occlusion, and severe jittering in the output. As a remedy, we propose POTR-3D, the first realization of a sequence-to-sequence 2D-to-3D lifting model for 3DMPPE, powered by a novel geometry-aware data augmentation strategy, capable of generating unbounded data with a variety of views while caring about the ground plane and occlusions. Through extensive experiments, we verify that the proposed model and data augmentation robustly generalizes to diverse unseen views, robustly recovers the poses against heavy occlusions, and reliably generates more natural and smoother outputs. The effectiveness of our approach is verified not only by achieving the state-of-the-art performance on public benchmarks, but also by qualitative results on more challenging in-the-wild videos. Demo videos are available at https://www.youtube.com/@potr3d.


title: Hawking temperature of black holes with multiple horizons

abstract: There are several well-established methods for computing thermodynamics in single-horizon spacetimes. However, understanding thermodynamics becomes particularly important when dealing with spacetimes with multiple horizons. Multiple horizons raise questions about the existence of a global temperature for such spacetimes. Recent studies highlight the significant role played by the contribution of all the horizons in determining Hawking temperature. Here we explore the Hawking temperature of a rotating and charged black hole in four spacetime dimensions and a rotating BTZ black hole. We also find that each horizon of those black holes contributes to the Hawking temperature. The effective Hawking temperature for a four-dimensional rotating and charged black hole depends only on its mass. This temperature is the same as the Hawking temperature of a Schwarzschild black hole. In contrast, the effective Hawking temperature depends on the black hole mass and angular momentum for a rotating BTZ hole.