title: Deep learning probability flows and entropy production rates in active matter

abstract: Active matter systems, from self-propelled colloids to motile bacteria, are characterized by the conversion of free energy into useful work at the microscopic scale. They involve physics beyond the reach of equilibrium statistical mechanics, and a persistent challenge has been to understand the nature of their nonequilibrium states. The entropy production rate and the probability current provide quantitative ways to do so by measuring the breakdown of time-reversal symmetry. Yet, their efficient computation has remained elusive, as they depend on the system's unknown and high-dimensional probability density. Here, building upon recent advances in generative modeling, we develop a deep learning framework to estimate the score of this density. We show that the score, together with the microscopic equations of motion, gives access to the entropy production rate, the probability current, and their decomposition into local contributions from individual particles. To represent the score, we introduce a novel, spatially-local transformer network architecture that learns high-order interactions between particles while respecting their underlying permutation symmetry. We demonstrate the broad utility and scalability of the method by applying it to several high-dimensional systems of active particles undergoing motility-induced phase separation (MIPS). We show that a single network trained on a system of 4096 particles at one packing fraction can generalize to other regions of the phase diagram, including systems with as many as 32768 particles. We use this observation to quantify the spatial structure of the departure from equilibrium in MIPS as a function of the number of particles and the packing fraction.

categories: cond-mat.stat-mech cond-mat.soft cs.LG cs.NA math.NA


title: On a new (21_4) polycyclic configuration

abstract: When searching for small 4-configurations of points and lines, polycyclic configurations, in which every symmetry class of points and lines contains the same number of elements, have proved to be quite useful. In this paper we construct and prove the existence of a previously unknown (21_4) conguration, which provides a counterexample to a conjecture of Branko Gr\"unbaum. In addition, we study some of its most important properties; in particular, we make a comparison with the well-known Gr\"unbaum-Rigby configuration. We show that there are exactly two (21_4) geometric polycyclic configurations and seventeen (21_4) combinatorial polycyclic configurations. We also discuss some possible generalizations.


title: Fourier inequalities in Morrey and Campanato spaces

abstract: We study norm inequalities for the Fourier transform, namely, \begin{equation}\label{introduction} \|\widehat f\|_{X_{p,q}^\lambda} \lesssim \|f\|_{Y}, \end{equation} where $X$ is either a Morrey or Campanato space and $Y$ is an appropriate function space. In the case of the Morrey space we sharpen the estimate $ \|\widehat f\|_{M_{p,q}^\lambda} \lesssim \|f\|_{L_{s',q}},$ $ s\geq 2,$ $\frac{1}{s} = \frac{1}{p}-\frac{\lambda}{n}.$ We also show that \eqref{introduction} does not hold when both $X$ and $Y$ are Morrey spaces. If $X$ is a Campanato space, we prove that \eqref{introduction} holds for $Y$ being the truncated Lebesgue space.


title: Smart Fuzzing of 5G Wireless Software Implementation

abstract: In this paper, we introduce a comprehensive approach to bolstering the security, reliability, and comprehensibility of OpenAirInterface5G (OAI5G), an open-source software framework for the exploration, development, and testing of 5G wireless communication systems. Firstly, we employ AFL++, a powerful fuzzing tool, to fuzzy-test OAI5G with respect to its configuration files rigorously. This extensive testing process helps identify errors, defects, and security vulnerabilities that may evade conventional testing methods. Secondly, we harness the capabilities of Large Language Models such as Google Bard to automatically decipher and document the meanings of parameters within the OAI5G codebase that are used in fuzzing. This automated parameter interpretation streamlines subsequent analyses and facilitates more informed decision-making. Together, these two techniques contribute to fortifying the OAI5G system, making it more robust, secure, and understandable for developers and analysts alike.


title: Study of neutron density fluctuation and neutron-proton correlation in Au+Au collisions using PYTHIA8/Angantyr

abstract: Utilizing the PYTHIA8 Angantyr model, which incorporates the multiple-parton interactions (MPI) based color reconnection (CR) mechanism, we study the relative neutron density fluctuation and neutron-proton correlation in Au+Au collisions at $\sqrt{s_\text{NN}}$ = 7.7, 11.5, 14.5, 19.6, 27, 39, 62.4, and 200 GeV. In this study, we have not only delved into the dependence of these two remarkable observations on rapidity, centrality, and energy, but also presented an analysis of their interplay with the MPI and CR. Our results have shown that the light nuclei yield ratio of proton, deuteron, and triton, expressed by the elegant expression $N_tN_p/N_d^2$, remains unchanged even as the rapidity coverage and collision centrality increase. Interestingly, we have also revealed that the effect of CR is entirely dependent on the presence of MPI; CR has no impact on the yield ratio if MPI is off. Our findings further demonstrate that the light nuclei yield ratio experiences a slight increase with increasing collision energy as predicted by the PYTHIA8 Angantyr model, but it cannot describe the non-monotonic trend observed by the STAR experiment. Based on the Angantyr model simulation results, it is essential not to overlook the correlation between neutron and proton fluctuations. The Angantyr model is a good baseline for studying collisions in the absence of a Quark-Gluon Plasma (QGP) system, given its lack of flow and jet quenching.


title: Point Cloud Network: An Order of Magnitude Improvement in Linear Layer Parameter Count

abstract: This paper introduces the Point Cloud Network (PCN) architecture, a novel implementation of linear layers in deep learning networks, and provides empirical evidence to advocate for its preference over the Multilayer Perceptron (MLP) in linear layers. We train several models, including the original AlexNet, using both MLP and PCN architectures for direct comparison of linear layers (Krizhevsky et al., 2012). The key results collected are model parameter count and top-1 test accuracy over the CIFAR-10 and CIFAR-100 datasets (Krizhevsky, 2009). AlexNet-PCN16, our PCN equivalent to AlexNet, achieves comparable efficacy (test accuracy) to the original architecture with a 99.5% reduction of parameters in its linear layers. All training is done on cloud RTX 4090 GPUs, leveraging pytorch for model construction and training. Code is provided for anyone to reproduce the trials from this paper.


title: Scaling Limits of the Wasserstein information matrix on Gaussian Mixture Models

abstract: We consider the Wasserstein metric on the Gaussian mixture models (GMMs), which is defined as the pullback of the full Wasserstein metric on the space of smooth probability distributions with finite second moment. It derives a class of Wasserstein metrics on probability simplices over one-dimensional bounded homogeneous lattices via a scaling limit of the Wasserstein metric on GMMs. Specifically, for a sequence of GMMs whose variances tend to zero, we prove that the limit of the Wasserstein metric exists after certain renormalization. Generalizations of this metric in general GMMs are established, including inhomogeneous lattice models whose lattice gaps are not the same, extended GMMs whose mean parameters of Gaussian components can also change, and the second-order metric containing high-order information of the scaling limit. We further study the Wasserstein gradient flows on GMMs for three typical functionals: potential, internal, and interaction energies. Numerical examples demonstrate the effectiveness of the proposed GMM models for approximating Wasserstein gradient flows.


title: Audience-specific Explanations for Machine Translation

abstract: In machine translation, a common problem is that the translation of certain words even if translated can cause incomprehension of the target language audience due to different cultural backgrounds. A solution to solve this problem is to add explanations for these words. In a first step, we therefore need to identify these words or phrases. In this work we explore techniques to extract example explanations from a parallel corpus. However, the sparsity of sentences containing words that need to be explained makes building the training dataset extremely difficult. In this work, we propose a semi-automatic technique to extract these explanations from a large parallel corpus. Experiments on English->German language pair show that our method is able to extract sentence so that more than 10% of the sentences contain explanation, while only 1.9% of the original sentences contain explanations. In addition, experiments on English->French and English->Chinese language pairs also show similar conclusions. This is therefore an essential first automatic step to create a explanation dataset. Furthermore we show that the technique is robust for all three language pairs.


title: Braid groups, elliptic curves, and resolving the quartic

abstract: We show that, up to a natural equivalence relation, the only non-trivial, non-identity holomorphic maps $\mathrm{Conf}_n\mathbb{C}\to\mathrm{Conf}_m\mathbb{C}$ between unordered configuration spaces, where $m\in\{3,4\}$, are the resolving quartic map $R\colon\mathrm{Conf}_4\mathbb{C}\to\mathrm{Conf}_3\mathbb{C}$, a map $\Psi_3\colon\mathrm{Conf}_3\mathbb{C}\to\mathrm{Conf}_4\mathbb{C}$ constructed from the inflection points of elliptic curves in a family, and $\Psi_3\circ R$. This completes the classification of holomorphic maps $\mathrm{Conf}_n\mathbb{C}\to\mathrm{Conf}_m\mathbb{C}$ for $m\leq n$, extending results of Lin, Chen and Salter, and partially resolves a conjecture of Farb. We also classify the holomorphic families of elliptic curves over $\mathrm{Conf}_n\mathbb{C}$. To do this we classify homomorphisms between braid groups with few strands and $\mathrm{PSL}_2\mathbb{Z}$, then apply powerful results from complex analysis and Teichm\"uller theory. Furthermore, we prove a conjecture of Castel about the equivalence classes of endomorphisms of the braid group with three strands.


title: Density engineering via inter-condensate dipole-dipole interactions

abstract: We study the effect of inter-condensate dipole-dipole interactions in a setup consisting of physically disconnected single-species dipolar Bose-Einstein condensates. In particular, making use of the long-range and anisotropic nature of dipole-dipole interactions, we show that the density of a {\em target} dipolar Bose-Einstein condensate can be axially confined and engineered using a trapped {\em control} dipolar condensate. Increasing the number of control condensates leads to exotic ground state structures, including periodic patterns. The latter consists of single and double-peaked structures with coherence between the peaks controlled via the separation between the control condensates, thereby accessing both supersolid-like structures and incoherent arrays of density peaks.


title: Joint $p$-Values for Higher-Powered Bayesian Model Checking with Frequentist Guarantees

abstract: We introduce a joint posterior $p$-value, an extension of the posterior predictive $p$-value for multiple test statistics, designed to address limitations of existing Bayesian $p$-values in the setting of continuous model expansion. In particular, we show that the posterior predictive $p$-value, as well as its sampled variant, become more conservative as the parameter dimension grows, and we demonstrate the ability of the joint $p$-value to overcome this problem in cases where we can select test statistics that are negatively associated under the posterior. We validate these conclusions with a pair of simulation examples in which the joint $p$-value achieves substantial gains to power with only a modest increase in computational cost.


title: Expressive variational quantum circuits provide inherent privacy in federated learning

abstract: Federated learning has emerged as a viable distributed solution to train machine learning models without the actual need to share data with the central aggregator. However, standard neural network-based federated learning models have been shown to be susceptible to data leakage from the gradients shared with the server. In this work, we introduce federated learning with variational quantum circuit model built using expressive encoding maps coupled with overparameterized ans\"atze. We show that expressive maps lead to inherent privacy against gradient inversion attacks, while overparameterization ensures model trainability. Our privacy framework centers on the complexity of solving the system of high-degree multivariate Chebyshev polynomials generated by the gradients of quantum circuit. We present compelling arguments highlighting the inherent difficulty in solving these equations, both in exact and approximate scenarios. Additionally, we delve into machine learning-based attack strategies and establish a direct connection between overparameterization in the original federated learning model and underparameterization in the attack model. Furthermore, we provide numerical scaling arguments showcasing that underparameterization of the expressive map in the attack model leads to the loss landscape being swamped with exponentially many spurious local minima points, thus making it extremely hard to realize a successful attack. This provides a strong claim, for the first time, that the nature of quantum machine learning models inherently helps prevent data leakage in federated learning.


title: Observation of light driven band structure via multi-band high harmonic spectroscopy

abstract: Intense light-matter interactions have revolutionized our ability to probe and manipulate quantum systems at sub-femtosecond time scales, opening routes to all-optical control of electronic currents in solids at petahertz rates. Such control typically requires electric field amplitudes $\sim V/\AA$, when the voltage drop across a lattice site becomes comparable to the characteristic band gap energies. In this regime, intense light-matter interaction induces significant modifications of electronic and optical properties, dramatically modifying the crystal band structure. Yet, identifying and characterizing such modifications remains an outstanding problem. As the oscillating electric field changes within the driving field's cycle, does the band-structure follow, and how can it be defined? Here we address this fundamental question, proposing all-optical spectroscopy to probe laser-induced closing of the band-gap between adjacent conduction bands. Our work reveals the link between nonlinear light matter interactions in strongly driven crystals and the sub-cycle modifications in their effective band structure.


title: Bound-state-in-continuum guided modes in a multilayer electro-optically active photonic integrated circuit platform

abstract: Bound states in the continuum (BICs) are localized states existing within a continuous spectrum of delocalized waves. Emerging multilayer photonic integrated circuit (PIC) platforms allow implementation of low index 1D guided modes within a high-index 2D slab mode continuum; however, conventional wisdom suggests that this always leads to large radiation losses. Here we demonstrate low-loss BIC guided modes for multiple mode polarizations and spatial orders in single- and multi-ridge low-index waveguides within a two-layer heterogeneously integrated electro-optically active photonic platform. The transverse electric (TE) polarized quasi-BIC guided mode with low, <1.4 dB/cm loss enables a Mach-Zehnder electro-optic amplitude modulator comprising a single straight Si3N4 ridge waveguide integrated with a continuous LiNbO3 slab layer. The abrupt optical transitions at the edges of the slab function as compact and efficient directional couplers eliminating the need for additional components. The modulator exhibits a low insertion loss of 2.3 dB and a high extinction ratio of 25 dB. The developed general theoretical model may enable innovative BIC-based approaches for important PIC functions, such as agile spectral filtering and switching, and may suggest new photonic architectures for quantum and neural network applications based on controlled interactions between multiple guided and delocalized modes.


title: Towards Counterfactual Fairness-aware Domain Generalization in Changing Environments

abstract: Recognizing the prevalence of domain shift as a common challenge in machine learning, various domain generalization (DG) techniques have been developed to enhance the performance of machine learning systems when dealing with out-of-distribution (OOD) data. Furthermore, in real-world scenarios, data distributions can gradually change across a sequence of sequential domains. While current methodologies primarily focus on improving model effectiveness within these new domains, they often overlook fairness issues throughout the learning process. In response, we introduce an innovative framework called Counterfactual Fairness-Aware Domain Generalization with Sequential Autoencoder (CDSAE). This approach effectively separates environmental information and sensitive attributes from the embedded representation of classification features. This concurrent separation not only greatly improves model generalization across diverse and unfamiliar domains but also effectively addresses challenges related to unfair classification. Our strategy is rooted in the principles of causal inference to tackle these dual issues. To examine the intricate relationship between semantic information, sensitive attributes, and environmental cues, we systematically categorize exogenous uncertainty factors into four latent variables: 1) semantic information influenced by sensitive attributes, 2) semantic information unaffected by sensitive attributes, 3) environmental cues influenced by sensitive attributes, and 4) environmental cues unaffected by sensitive attributes. By incorporating fairness regularization, we exclusively employ semantic information for classification purposes. Empirical validation on synthetic and real-world datasets substantiates the effectiveness of our approach, demonstrating improved accuracy levels while ensuring the preservation of fairness in the evolving landscape of continuous domains.


title: Deep3DSketch+: Rapid 3D Modeling from Single Free-hand Sketches

abstract: The rapid development of AR/VR brings tremendous demands for 3D content. While the widely-used Computer-Aided Design (CAD) method requires a time-consuming and labor-intensive modeling process, sketch-based 3D modeling offers a potential solution as a natural form of computer-human interaction. However, the sparsity and ambiguity of sketches make it challenging to generate high-fidelity content reflecting creators' ideas. Precise drawing from multiple views or strategic step-by-step drawings is often required to tackle the challenge but is not friendly to novice users. In this work, we introduce a novel end-to-end approach, Deep3DSketch+, which performs 3D modeling using only a single free-hand sketch without inputting multiple sketches or view information. Specifically, we introduce a lightweight generation network for efficient inference in real-time and a structural-aware adversarial training approach with a Stroke Enhancement Module (SEM) to capture the structural information to facilitate learning of the realistic and fine-detailed shape structures for high-fidelity performance. Extensive experiments demonstrated the effectiveness of our approach with the state-of-the-art (SOTA) performance on both synthetic and real datasets.


title: ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs

abstract: Large Language Models (LLMs) still struggle with natural language reasoning tasks. Motivated by the society of minds (Minsky, 1988), we propose ReConcile, a multi-model multi-agent framework designed as a round table conference among diverse LLM agents. ReConcile enhances collaborative reasoning between LLM agents via multiple rounds of discussion, learning to convince other agents to improve their answers, and employing a confidence-weighted voting mechanism that leads to a better consensus. In each round, ReConcile initiates discussion between agents via a 'discussion prompt' that consists of (a) grouped answers and explanations generated by each agent in the previous round, (b) their confidence scores, and (c) demonstrations of answer-rectifying human explanations, used for convincing other agents. Experiments on seven benchmarks demonstrate that ReConcile significantly improves LLMs' reasoning -- both individually and as a team -- surpassing prior single-agent and multi-agent baselines by up to 11.4% and even outperforming GPT-4 on three datasets. ReConcile also flexibly incorporates different combinations of agents, including API-based, open-source, and domain-specific models, leading to an 8% improvement on MATH. Finally, we analyze the individual components of ReConcile, demonstrating that the diversity originating from different models is critical to its superior performance. Code: https://github.com/dinobby/ReConcile


title: PEARLS: JWST counterparts of micro-Jy radio sources in the Time Domain Field

abstract: The Time Domain Field (TDF) near the North Ecliptic Pole in JWST's continuous-viewing zone will become a premier "blank field" for extragalactic science. JWST/NIRCam data in a 16 arcmin$^2$ portion of the TDF identify 4.4 $\mu$m counterparts for 62 of 63 3 GHz sources with S(3 GHz) > 5 {\mu}Jy. The one unidentified radio source may be a lobe of a nearby Seyfert galaxy, or it may be an infrared-faint radio source. The bulk properties of the radio-host galaxies are consistent with those found by previous work: redshifts range from 0.14 to 4.4 with a median redshift of 1.33. The radio emission arises primarily from star formation in $\sim 2/3$ of the sample and from an active galactic nucleus in $\sim 1/3$, but just over half the sample shows evidence for an AGN either in the spectral energy distribution or by radio excess. All but three counterparts are brighter than magnitude 23 AB at 4.4 $\mu$m, and the exquisite resolution of JWST identifies correct counterparts for sources for which observations with lower angular resolution would mis-identify a nearby bright source as the counterpart when the correct one is faint and red. Up to 11% of counterparts might have been unidentified or misidentified absent NIRCam observations.


title: Mass equidistribution for Saito-Kurokawa lifts

abstract: Let $F$ be a holomorphic cuspidal Hecke eigenform for $\mathrm{Sp}_4(\mathbb{Z})$ of weight $k$ that is a Saito--Kurokawa lift. Assuming the Generalized Riemann Hypothesis (GRH), we prove that the mass of $F$ equidistributes on the Siegel modular variety as $k\longrightarrow \infty$. As a corollary, we show under GRH that the zero divisors of Saito--Kurokawa lifts equidistribute as their weights tend to infinity.


title: Holomorphic discs of negative Maslov index and extended deformations in mirror symmetry

abstract: The SYZ approach to mirror symmetry for log Calabi-Yau manifolds starts from a Lagrangian torus fibration on the complement of an anticanonical divisor. A mirror space is constructed by gluing local charts (moduli spaces of local systems on generic torus fibers) via wall-crossing transformations which account for corrections to the analytic structure of moduli spaces of objects of the Fukaya category induced by bubbling of Maslov index 0 holomorphic discs, and made into a Landau-Ginzburg model by equipping it with a regular function (the superpotential) which enumerates Maslov index 2 holomorphic discs. When they occur, holomorphic discs of negative Maslov index deform this picture by introducing inconsistencies in the wall-crossing transformations, so that the mirror is no longer an analytic space; the geometric features of the corrected mirror can be understood in the language of extended deformations of Landau-Ginzburg models. We illustrate this phenomenon (and show that it actually occurs) by working through the construction for an explicit example (a log Calabi-Yau 4-fold obtained by blowing up a toric variety), and discuss a family Floer approach to the geometry of the corrected mirror in this setting. Along the way, we introduce a Morse-theoretic model for family Floer theory which may be of independent interest.


title: JWST NIRCam Observations of SN 1987A: Spitzer Comparison and Spectral Decomposition

abstract: JWST NIRCam observations at 1.5-4.5 $\mu$m have provided broad and narrow band imaging of the evolving remnant of SN 1987A with unparalleled sensitivity and spatial resolution. Comparing with previous marginally spatially resolved Spitzer IRAC observations from 2004-2019 confirms that the emission arises from the circumstellar equatorial ring (ER), and the current brightness at 3.6 and 4.5 $\mu$m was accurately predicted by extrapolation of the declining brightness tracked by IRAC. Despite the regular light curve, the NIRCam observations clearly reveal that much of this emission is from a newly developing outer portion of the ER. Spots in the outer ER tend to lie at position angles in between the well-known ER hotspots. We show that the bulk of the emission in the field can be represented by 5 standard spectral energy distributions (SEDs), each with a distinct origin and spatial distribution. This spectral decomposition provides a powerful technique for distinguishing overlapping emission from the circumstellar medium (CSM) and the supernova (SN) ejecta, excited by the forward and reverse shocks respectively.


title: Electric Autonomous Mobility-on-Demand: Jointly Optimal Vehicle Design and Fleet Operation

abstract: The advent of autonomous driving and electrification is enabling the deployment of Electric Autonomous Mobility-on-Demand (E-AMoD) systems, whereby electric autonomous vehicles provide on-demand mobility. Crucially, the design of the individual vehicles and the fleet, and the operation of the system are strongly coupled. Hence, to maximize the system-level performance, they must be optimized in a joint fashion. To this end, this paper presents a framework to jointly optimize the fleet design in terms of battery capacity and number of vehicles, and the operational strategies of the E-AMoD system, with the aim of maximizing the operator's total profit. Specifically, we first formulate this joint optimization problem using directed acyclic graphs as a mixed integer linear program, which can be solved using commercial solvers with optimality guarantees. Second, to solve large instances of the problem, we propose a solution algorithm that solves for randomly sampled sub-problems, providing a more conservative solution of the full problem, and devise a heuristic approach to tackle larger individual sub-problem instances. Finally, we showcase our framework on a real-world case study in Manhattan, where we demonstrate the interdependence between the number of vehicles, their battery size, and operational and fixed costs. Our results indicate that to maximize a mobility operator's profit, a fleet of small and light vehicles with battery capacity of 20 kWh only can strike the best trade-off in terms of battery degradation, fixed costs and operational efficiency.


title: Performance Analysis of UNet and Variants for Medical Image Segmentation

abstract: Medical imaging plays a crucial role in modern healthcare by providing non-invasive visualisation of internal structures and abnormalities, enabling early disease detection, accurate diagnosis, and treatment planning. This study aims to explore the application of deep learning models, particularly focusing on the UNet architecture and its variants, in medical image segmentation. We seek to evaluate the performance of these models across various challenging medical image segmentation tasks, addressing issues such as image normalization, resizing, architecture choices, loss function design, and hyperparameter tuning. The findings reveal that the standard UNet, when extended with a deep network layer, is a proficient medical image segmentation model, while the Res-UNet and Attention Res-UNet architectures demonstrate smoother convergence and superior performance, particularly when handling fine image details. The study also addresses the challenge of high class imbalance through careful preprocessing and loss function definitions. We anticipate that the results of this study will provide useful insights for researchers seeking to apply these models to new medical imaging problems and offer guidance and best practices for their implementation.


title: Completeness of qufinite ZXW calculus, a graphical language for finite-dimensional quantum theory

abstract: Finite-dimensional quantum theory serves as the theoretical foundation for quantum information and computation. Mathematically, it is formalized in the category FHilb, comprising all finite-dimensional Hilbert spaces and linear maps between them. However, there has not been a graphical language for FHilb which is both universal and complete and thus incorporates a set of rules rich enough to derive any equality of the underlying formalism solely by rewriting. In this paper, we introduce the qufinite ZXW calculus - a graphical language for reasoning about finite-dimensional quantum theory. We set up a unique normal form to represent an arbitrary tensor and prove the completeness of this calculus by demonstrating that any qufinite ZXW diagram can be rewritten into its normal form. This result implies the equivalence of the qufinite ZXW calculus and the category FHilb, leading to a purely diagrammatic framework for finite-dimensional quantum theory with the same reasoning power. In addition, we identify several domains where the application of the qufinite ZXW calculus holds promise. These domains include spin networks, interacting mixed-dimensional systems in quantum chemistry, quantum programming, high-level description of quantum algorithms, and mixed-dimensional quantum computing. Our work paves the way for a comprehensive diagrammatic description of quantum physics, opening the doors of this area to the wider public.


title: Efficient N:M Sparse DNN Training Using Algorithm, Architecture, and Dataflow Co-Design

abstract: Sparse training is one of the promising techniques to reduce the computational cost of DNNs while retaining high accuracy. In particular, N:M fine-grained structured sparsity, where only N out of consecutive M elements can be nonzero, has attracted attention due to its hardware-friendly pattern and capability of achieving a high sparse ratio. However, the potential to accelerate N:M sparse DNN training has not been fully exploited, and there is a lack of efficient hardware supporting N:M sparse training. To tackle these challenges, this paper presents a computation-efficient training scheme for N:M sparse DNNs using algorithm, architecture, and dataflow co-design. At the algorithm level, a bidirectional weight pruning method, dubbed BDWP, is proposed to leverage the N:M sparsity of weights during both forward and backward passes of DNN training, which can significantly reduce the computational cost while maintaining model accuracy. At the architecture level, a sparse accelerator for DNN training, namely SAT, is developed to neatly support both the regular dense operations and the computation-efficient N:M sparse operations. At the dataflow level, multiple optimization methods ranging from interleave mapping, pre-generation of N:M sparse weights, and offline scheduling, are proposed to boost the computational efficiency of SAT. Finally, the effectiveness of our training scheme is evaluated on a Xilinx VCU1525 FPGA card using various DNN models and datasets. Experimental results show the SAT accelerator with the BDWP sparse training method under 2:8 sparse ratio achieves an average speedup of 1.75x over that with the dense training, accompanied by a negligible accuracy loss of 0.56% on average. Furthermore, our proposed training scheme significantly improves the training throughput by 2.97~25.22x and the energy efficiency by 1.36~3.58x over prior FPGA-based accelerators.


title: Understanding Deep Gradient Leakage via Inversion Influence Functions

abstract: Deep Gradient Leakage (DGL) is a highly effective attack that recovers private training images from gradient vectors. This attack casts significant privacy challenges on distributed learning from clients with sensitive data, where clients are required to share gradients. Defending against such attacks requires but lacks an understanding of when and how privacy leakage happens, mostly because of the black-box nature of deep networks. In this paper, we propose a novel Inversion Influence Function (I$^2$F) that establishes a closed-form connection between the recovered images and the private gradients by implicitly solving the DGL problem. Compared to directly solving DGL, I$^2$F is scalable for analyzing deep networks, requiring only oracle access to gradients and Jacobian-vector products. We empirically demonstrate that I$^2$F effectively approximated the DGL generally on different model architectures, datasets, modalities, attack implementations, and perturbation-based defenses. With this novel tool, we provide insights into effective gradient perturbation directions, the unfairness of privacy protection, and privacy-preferred model initialization. Our codes are provided in https://github.com/illidanlab/inversion-influence-function.


title: Splittings for symbolic powers of edge ideals of complete graphs

abstract: In this paper we study the $s$-th symbolic powers of the edge ideals of complete graphs. In particular, we provide a criterion for finding an Eliahou-Kervaire splitting on these ideals, and use the splitting to provide a description for the graded Betti numbers. We also discuss the symbolic powers and graded Betti numbers of edge ideals of parallelizations of finite simple graphs.


title: Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient Pruning of A Multilingual ASR Model

abstract: Neural network pruning offers an effective method for compressing a multilingual automatic speech recognition (ASR) model with minimal performance loss. However, it entails several rounds of pruning and re-training needed to be run for each language. In this work, we propose the use of an adaptive masking approach in two scenarios for pruning a multilingual ASR model efficiently, each resulting in sparse monolingual models or a sparse multilingual model (named as Dynamic ASR Pathways). Our approach dynamically adapts the sub-network, avoiding premature decisions about a fixed sub-network structure. We show that our approach outperforms existing pruning methods when targeting sparse monolingual models. Further, we illustrate that Dynamic ASR Pathways jointly discovers and trains better sub-networks (pathways) of a single multilingual model by adapting from different sub-network initializations, thereby reducing the need for language-specific pruning.


title: Differential Evolution Algorithm Based Hyperparameter Selection of Gated Recurrent Unit for Electrical Load Forecasting

abstract: Accurate load forecasting remains a formidable challenge in numerous sectors, given the intricate dynamics of dynamic power systems, which often defy conventional statistical models. As a response, time-series methodologies like ARIMA and sophisticated deep learning techniques such as Artificial Neural Networks (ANN) and Long Short-Term Memory (LSTM) networks have demonstrated their mettle by achieving enhanced predictive performance. In our investigation, we delve into the efficacy of the relatively recent Gated Recurrent Network (GRU) model within the context of load forecasting. GRU models are garnering attention due to their inherent capacity to adeptly capture and model temporal dependencies within data streams. Our methodology entails harnessing the power of Differential Evolution, a versatile optimization technique renowned for its prowess in delivering scalable, robust, and globally optimal solutions, especially in scenarios involving non-differentiable, multi-objective, or constrained optimization challenges. Through rigorous analysis, we undertake a comparative assessment of the proposed Gated Recurrent Network model, collaboratively fused with various metaheuristic algorithms, evaluating their performance by leveraging established numerical benchmarks such as Mean Squared Error (MSE) and Mean Absolute Percentage Error (MAPE). Our empirical investigations are conducted using power load data originating from the Ontario province, Canada. Our research findings cast a spotlight on the remarkable potential of metaheuristic-augmented Gated Recurrent Network models in substantially augmenting load forecasting precision, offering tailored, optimal hyperparameter configurations uniquely suited to each model's performance characteristics.


title: Annealed local limit theorem for Sinai's random walk in random environment

abstract: We consider Sinai's random walk in random environment $(S_n)_{n\in\mathbb{N}}$. We prove a local limit theorem for $(S_n)_{n\in\mathbb{N}}$ under the annealed law $\mathbb{P}$. As a consequence, we get an equivalent for the annealed probability $\mathbb{P}(S_n=z_n)$ as $n$ goes to infinity, when $z_n=O\big((\log n)^2\big)$. To this aim, we develop a path decomposition for the potential of Sinai's walk, that is, for some random walks with i.i.d. increments. The proof also relies on renewal theory, a coupling argument, a very careful analysis of the environments and trajectories of Sinai's walk satisfying $S_n=z_n$, and on precise estimates for random walks conditioned to stay positive or nonnegative.


title: A Hybrid Deep Learning-based Approach for Optimal Genotype by Environment Selection

abstract: Precise crop yield prediction is essential for improving agricultural practices and ensuring crop resilience in varying climates. Integrating weather data across the growing season, especially for different crop varieties, is crucial for understanding their adaptability in the face of climate change. In the MLCAS2021 Crop Yield Prediction Challenge, we utilized a dataset comprising 93,028 training records to forecast yields for 10,337 test records, covering 159 locations across 28 U.S. states and Canadian provinces over 13 years (2003-2015). This dataset included details on 5,838 distinct genotypes and daily weather data for a 214-day growing season, enabling comprehensive analysis. As one of the winning teams, we developed two novel convolutional neural network (CNN) architectures: the CNN-DNN model, combining CNN and fully-connected networks, and the CNN-LSTM-DNN model, with an added LSTM layer for weather variables. Leveraging the Generalized Ensemble Method (GEM), we determined optimal model weights, resulting in superior performance compared to baseline models. The GEM model achieved lower RMSE (5.55% to 39.88%), reduced MAE (5.34% to 43.76%), and higher correlation coefficients (1.1% to 10.79%) when evaluated on test data. We applied the CNN-DNN model to identify top-performing genotypes for various locations and weather conditions, aiding genotype selection based on weather variables. Our data-driven approach is valuable for scenarios with limited testing years. Additionally, a feature importance analysis using RMSE change highlighted the significance of location, MG, year, and genotype, along with the importance of weather variables MDNI and AP.


title: Graph Neural Network for Stress Predictions in Stiffened Panels Under Uniform Loading

abstract: Machine learning (ML) and deep learning (DL) techniques have gained significant attention as reduced order models (ROMs) to computationally expensive structural analysis methods, such as finite element analysis (FEA). Graph neural network (GNN) is a particular type of neural network which processes data that can be represented as graphs. This allows for efficient representation of complex geometries that can change during conceptual design of a structure or a product. In this study, we propose a novel graph embedding technique for efficient representation of 3D stiffened panels by considering separate plate domains as vertices. This approach is considered using Graph Sampling and Aggregation (GraphSAGE) to predict stress distributions in stiffened panels with varying geometries. A comparison between a finite-element-vertex graph representation is conducted to demonstrate the effectiveness of the proposed approach. A comprehensive parametric study is performed to examine the effect of structural geometry on the prediction performance. Our results demonstrate the immense potential of graph neural networks with the proposed graph embedding method as robust reduced-order models for 3D structures.


title: Local infrared safety in time-ordered perturbation theory

abstract: We develop a general expression for weighted cross sections in leptonic annihilation to hadrons based on time-ordered perturbation theory (TOPT). The analytic behavior of the resulting integrals over spatial momenta can be analyzed in the language of Landau equations and infrared (IR) power counting. For any infrared-safe weight, the cancellation of infrared divergences is implemented locally at the integrand level, and in principle can be evaluated numerically in four dimensions. We go on to show that it is possible to eliminate unphysical singularities that appear in time-ordered perturbation theory for arbitrary amplitudes. This is done by reorganizing TOPT into an equivalent form that combines classes of time orderings into a ``partially time-ordered perturbation theory". Applying the formalism to leptonic annihilation, we show how to derive diagrammatic expressions with only physical unitarity cuts.


title: Zeroth-Order Federated Methods for Stochastic MPECs and Nondifferentiable Nonconvex Hierarchical Optimization

abstract: Motivated by the emergence of federated learning (FL), we design and analyze federated methods for addressing: (i) Nondifferentiable nonconvex optimization; (ii) Bilevel optimization; (iii) Minimax problems; and (iv) Two-stage stochastic mathematical programs with equilibrium constraints (2s-SMPEC). Research on these problems has been limited and afflicted by reliance on strong assumptions, including the need for differentiability of the implicit function and the absence of constraints in the lower-level problem, among others. We make the following contributions. In (i), by leveraging convolution-based smoothing and Clarke's subdifferential calculus, we devise a randomized smoothing-enabled zeroth-order FL method and derive communication and iteration complexity guarantees for computing an approximate Clarke stationary point. To contend with (ii) and (iii), we devise a unifying randomized implicit zeroth-order FL framework, equipped with explicit communication and iteration complexities. Importantly, our method utilizes delays during local steps to skip calls to the inexact lower-level FL oracle. This results in significant reduction in communication overhead. In (iv), we devise an inexact implicit variant of the method in (i). Remarkably, this method achieves a total communication complexity matching that of single-level nonsmooth nonconvex optimization in FL. We empirically validate the theoretical findings on instances of federated nonsmooth and hierarchical problems.


title: Optical Cluster Cosmology with SDSS redMaPPer clusters and HSC-Y3 lensing measurements

abstract: We present cosmology results obtained from a blind joint analysis of the abundance, projected clustering, and weak lensing of galaxy clusters measured from the Sloan Digital Sky Survey (SDSS) redMaPPer cluster catalog and the Hyper-Suprime Cam (HSC) Year3 shape catalog. We present a full-forward model for the cluster observables, which includes empirical modeling for the anisotropic boosts on the lensing and clustering signals of optical clusters. We validate our analysis via mock cluster catalogs which include observational systematics, such as the projection effect and the effect of baryonic feedback, and find that our analysis can robustly constrain cosmological parameters in an unbiased manner without any informative priors on our model parameters. The joint analysis of our observables in the context of the flat $\Lambda$CDM model results in cosmological constraints for $S_8\equiv \sigma_8 \sqrt{\Omega_{\rm m} / 0.3}=0.816^{+0.041}_{-0.039}$. Our result is consistent with the $S_8$ inference from other cosmic microwave background- and large scale structure-based cosmology analyses, including the result from the \emph{Planck} 2018 primary CMB analysis.


title: Directed propaganda in the majority-rule model

abstract: Advertisement and propaganda have changed continuously in the past decades, mainly due to the people's interactions at online platforms and social networks, and operate nowadays reaching a highly specific online audience instead targeting the masses. The impacts of this new media effect, oriented directly for a specific audience, is investigated on this study, in which we focus on the opinion evolution of agents in the majority-rule model, considering the presence of directed propaganda. We introduce $p$ as the probability of a "positive" external propaganda and $q$ as the probability to the agents follow the external propaganda. Our results show that the usual majority-rule model stationary state is reached, with a full consensus, only for two cases, namely when the external propaganda is absent or when the media favors only one of the two opinions. However, even for a small influence of external propaganda, the final state is reached with a majority opinion dominating the population. For the case in which the propaganda influence is strong enough among the agents, we show that the consensus can not be reached at all, and we observe the polarization of opinions. In addition, we show through analytical and numerical results that the system undergoes an order-disorder phase transition that occurs at $q_c = 1/3$ for the case $p = 0.5$.


title: On the generalized Tur\'an problem for odd cycles

abstract: In 1984, Erd\H{o}s conjectured that the number of pentagons in any triangle-free graph on $n$ vertices is at most $(n/5)^5$, which is sharp by the balanced blow-up of a pentagon. This was proved by Grzesik, and independently by Hatami, Hladk\'y, Kr\'al', Norine and Razborov. As an extension of this result for longer cycles, we prove that for each odd $k\geq 7$, the balanced blow-up of $C_k$ (uniquely) maximises the number of $k$-cycles among $C_{k-2}$-free graphs on $n$ vertices, as long as $n$ is sufficiently large. We also show that this is no longer true if $n$ is not assumed to be sufficiently large. Our result strengthens results of Grzesik and Kielak who proved that for each odd $k\geq 7$, the balanced blow-up of $C_k$ maximises the number of $k$-cycles among graphs with a given number of vertices and no odd cycles of length less than $k$. We further show that if $k$ and $\ell$ are odd and $k$ is sufficiently large compared to $\ell$, then the balanced blow-up of $C_{\ell+2}$ does not asymptotically maximise the number of $k$-cycles among $C_{\ell}$-free graphs on $n$ vertices. This disproves a conjecture of Grzesik and Kielak.


title: Minimization of energy functionals via FEM: implementation of hp-FEM

abstract: Many problems in science and engineering can be rigorously recast into minimizing a suitable energy functional. We have been developing efficient and flexible solution strategies to tackle various minimization problems by employing finite element discretization with P1 triangular elements [1,2]. An extension to rectangular hp-finite elements in 2D is introduced in this contribution.


title: Memory-augmented conformer for improved end-to-end long-form ASR

abstract: Conformers have recently been proposed as a promising modelling approach for automatic speech recognition (ASR), outperforming recurrent neural network-based approaches and transformers. Nevertheless, in general, the performance of these end-to-end models, especially attention-based models, is particularly degraded in the case of long utterances. To address this limitation, we propose adding a fully-differentiable memory-augmented neural network between the encoder and decoder of a conformer. This external memory can enrich the generalization for longer utterances since it allows the system to store and retrieve more information recurrently. Notably, we explore the neural Turing machine (NTM) that results in our proposed Conformer-NTM model architecture for ASR. Experimental results using Librispeech train-clean-100 and train-960 sets show that the proposed system outperforms the baseline conformer without memory for long utterances.


title: A numerical framework for simulating progressive failure in composite laminates under high-cycle fatigue loading

abstract: In this work, a recently proposed high-cycle fatigue cohesive zone model, which covers crack initiation and propagation with limited input parameters, is embedded in a robust and efficient numerical framework for simulating progressive failure in composite laminates under fatigue loading. The fatigue cohesive zone model is enhanced with an implicit time integration scheme of the fatigue damage variable which allows for larger cycle increments and more efficient analyses. The method is combined with an adaptive strategy for determining the cycle increment based on global convergence rates. Moreover, a consistent material tangent stiffness matrix has been derived by fully linearizing the underlying mixed-mode quasi-static model and the fatigue damage update. The enhanced fatigue cohesive zone model is used to describe matrix cracking and delamination in laminates. In order to allow for matrix cracks to initiate at arbitrary locations and to avoid complex and costly mesh generation, the phantom node version of the eXtended finite element method (XFEM) is employed. For the insertion of new crack segments, an XFEM fatigue crack insertion criterion is presented, which is consistent with the fatigue cohesive zone formulation. It is shown with numerical examples that the improved fatigue damage update enhances the accuracy, efficiency and robustness of the numerical simulations significantly. The numerical framework is applied to the simulation of progressive fatigue failure in an open-hole [$\pm$45]-laminate. It is demonstrated that the numerical model is capable of accurately and efficiently simulating the complete failure process from distributed damage to localized failure.


title: Anti-It\^o noise-induced phase transitions in tumor growth with chemotherapy

abstract: The objective of this work is to apply the H\"anggi-Klimontovich stochastic differential equations to model and study the effects of anti-tumor chemotherapy in the case of continuous infusion delivering. The fluctuations generated by variations in drug concentration are modeled by the H\"anggi-Klimontovich stochastic integral. This integral, which in the physics literature is sometimes called anti-It\^o integral, in the last decade it has been referenced quite as the more appropriate stochastic integral for model various biological and physical systems. Then, we make some comparisons with the model based on It\^o stochastic differential equations and their phase transitions that they generate, showing that the H\"anggi-Klimontovich stochastic differential equations lead to more biologically realistic results.


title: Modelling, Simulation, and Control of a Flexible Space Launch Vehicle

abstract: Modern Space Launch Vehicles (SLVs), being slender in shape and due to the use of lightweight materials, are generally flexible in nature. This structural flexibility, when coupled with sensor and actuator dynamics, can adversely affect the control of SLV, which may lead to vehicle instability and, in the worst-case scenario, to structural failure. This work focuses on modelling and simulation of rigid and flexible dynamics of an SLV and its interactions with the control system. SpaceX's Falcon 9 has been selected for this study. The flexible modes are calculated using modal analysis in Ansys. High-fidelity nonlinear simulation is developed which incorporates the flexible modes and their interactions with rigid degrees of freedom. Moreover, linearized models are developed for flexible body dynamics, over the complete trajectory until the first stage's separation. Using classical control methods, attitude controllers, that keep the SLV on its desired trajectory, are developed, and multiple filters are designed to suppress the interactions of flexible dynamics. The designed controllers along with filters are implemented in the nonlinear simulation. Furthermore, to demonstrate the robustness of designed controllers, Monte-Carlo simulations are carried out and results are presented.


title: Robust Stability Analysis of a Class of LTV Systems

abstract: Many physical systems are inherently time-varying in nature. When these systems are linearized around a trajectory, generally, the resulting system is Linear Time-Varying (LTV). LTV systems describe an important class of linear systems and can be thought of as a natural extension of LTI systems. However, it is well known that, unlike LTI systems, the eigenvalues of an LTV system do not determine its stability. In this paper, the stability conditions for a class of LTV systems are derived. This class is composed of piecewise LTV systems, i.e. LTV systems that are piecewise linear in time. Sufficient conditions of stability are derived in the form of linear matrix inequalities (LMIs) by using the Lyapunov stability criterion. The feasibility of LMIs guarantees the stability of a given piecewise LTV system. Furthermore, uncertain piecewise LTV systems with scalar parametric uncertainty are also considered. Sufficient conditions for robust stability of this case are also presented, which come out to be quasi-LMIs, which can be optimized using a bisection algorithm to find the bounds of uncertainty for which the system is stable. The proposed method is applied to the problem of pitch angle control of a space launch vehicle, and the results are presented.


title: Tuples of homological invariants of edge ideals

abstract: Let $G$ be a graph and $I(G)$ its edge ideal. In this paper, we completely determine the tuples $(\dim R/I(G), \depth (R/I(G)), \reg (R/I(G)))$ when the number of vertices is fixed for any graphs $G$.


title: PyPose v0.6: The Imperative Programming Interface for Robotics

abstract: PyPose is an open-source library for robot learning. It combines a learning-based approach with physics-based optimization, which enables seamless end-to-end robot learning. It has been used in many tasks due to its meticulously designed application programming interface (API) and efficient implementation. From its initial launch in early 2022, PyPose has experienced significant enhancements, incorporating a wide variety of new features into its platform. To satisfy the growing demand for understanding and utilizing the library and reduce the learning curve of new users, we present the fundamental design principle of the imperative programming interface, and showcase the flexible usage of diverse functionalities and modules using an extremely simple Dubins car example. We also demonstrate that the PyPose can be easily used to navigate a real quadruped robot with a few lines of code.


title: Encoding optimization for quantum machine learning demonstrated on a superconducting transmon qutrit

abstract: Qutrits, three-level quantum systems, have the advantage of potentially requiring fewer components than the typically used two-level qubits to construct equivalent quantum circuits. This work investigates the potential of qutrit parametric circuits in machine learning classification applications. We propose and evaluate different data-encoding schemes for qutrits, and find that the classification accuracy varies significantly depending on the used encoding. We therefore propose a training method for encoding optimization that allows to consistently achieve high classification accuracy. Our theoretical analysis and numerical simulations indicate that the qutrit classifier can achieve high classification accuracy using fewer components than a comparable qubit system. We showcase the qutrit classification using the optimized encoding method on superconducting transmon qutrits, demonstrating the practicality of the proposed method on noisy hardware. Our work demonstrates high-precision ternary classification using fewer circuit elements, establishing qutrit parametric quantum circuits as a viable and efficient tool for quantum machine learning applications.


title: GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators

abstract: Imitation learning from human demonstrations is a powerful framework to teach robots new skills. However, the performance of the learned policies is bottlenecked by the quality, scale, and variety of the demonstration data. In this paper, we aim to lower the barrier to collecting large and high-quality human demonstration data by proposing GELLO, a general framework for building low-cost and intuitive teleoperation systems for robotic manipulation. Given a target robot arm, we build a GELLO controller that has the same kinematic structure as the target arm, leveraging 3D-printed parts and off-the-shelf motors. GELLO is easy to build and intuitive to use. Through an extensive user study, we show that GELLO enables more reliable and efficient demonstration collection compared to commonly used teleoperation devices in the imitation learning literature such as VR controllers and 3D spacemouses. We further demonstrate the capabilities of GELLO for performing complex bi-manual and contact-rich manipulation tasks. To make GELLO accessible to everyone, we have designed and built GELLO systems for 3 commonly used robotic arms: Franka, UR5, and xArm. All software and hardware are open-sourced and can be found on our website: https://wuphilipp.github.io/gello/.


title: Privacy Assessment on Reconstructed Images: Are Existing Evaluation Metrics Faithful to Human Perception?

abstract: Hand-crafted image quality metrics, such as PSNR and SSIM, are commonly used to evaluate model privacy risk under reconstruction attacks. Under these metrics, reconstructed images that are determined to resemble the original one generally indicate more privacy leakage. Images determined as overall dissimilar, on the other hand, indicate higher robustness against attack. However, there is no guarantee that these metrics well reflect human opinions, which, as a judgement for model privacy leakage, are more trustworthy. In this paper, we comprehensively study the faithfulness of these hand-crafted metrics to human perception of privacy information from the reconstructed images. On 5 datasets ranging from natural images, faces, to fine-grained classes, we use 4 existing attack methods to reconstruct images from many different classification models and, for each reconstructed image, we ask multiple human annotators to assess whether this image is recognizable. Our studies reveal that the hand-crafted metrics only have a weak correlation with the human evaluation of privacy leakage and that even these metrics themselves often contradict each other. These observations suggest risks of current metrics in the community. To address this potential risk, we propose a learning-based measure called SemSim to evaluate the Semantic Similarity between the original and reconstructed images. SemSim is trained with a standard triplet loss, using an original image as an anchor, one of its recognizable reconstructed images as a positive sample, and an unrecognizable one as a negative. By training on human annotations, SemSim exhibits a greater reflection of privacy leakage on the semantic level. We show that SemSim has a significantly higher correlation with human judgment compared with existing metrics. Moreover, this strong correlation generalizes to unseen datasets, models and attack methods.


title: NeRRF: 3D Reconstruction and View Synthesis for Transparent and Specular Objects with Neural Refractive-Reflective Fields

abstract: Neural radiance fields (NeRF) have revolutionized the field of image-based view synthesis. However, NeRF uses straight rays and fails to deal with complicated light path changes caused by refraction and reflection. This prevents NeRF from successfully synthesizing transparent or specular objects, which are ubiquitous in real-world robotics and A/VR applications. In this paper, we introduce the refractive-reflective field. Taking the object silhouette as input, we first utilize marching tetrahedra with a progressive encoding to reconstruct the geometry of non-Lambertian objects and then model refraction and reflection effects of the object in a unified framework using Fresnel terms. Meanwhile, to achieve efficient and effective anti-aliasing, we propose a virtual cone supersampling technique. We benchmark our method on different shapes, backgrounds and Fresnel terms on both real-world and synthetic datasets. We also qualitatively and quantitatively benchmark the rendering results of various editing applications, including material editing, object replacement/insertion, and environment illumination estimation. Codes and data are publicly available at https://github.com/dawning77/NeRRF.


title: Pairing dome from an emergent Feshbach resonance in a strongly repulsive bilayer model

abstract: A key to understanding unconventional superconductivity lies in unraveling the pairing mechanism of mobile charge carriers in doped antiferromagnets, yielding an effective attraction between charges even in the presence of strong repulsive Coulomb interactions. Here, we study pairing in a mixed-dimensional (mixD) $t-J$ model, featuring robust binding energies -- despite dominant repulsive interactions -- that are strongly enhanced in the finite doping regime. The single and coupled mixD ladders we study, corresponding to bilayers of width $w\leq 2$, feature a crossover from tightly bound pairs of holes (closed channel) at small repulsion, to more spatially extended, correlated pairs of individual holes (open channel) at large repulsion. We derive an effective model for the latter, in which the attraction is mediated by the closed channel, in analogy to atomic Feshbach resonances. Using density matrix renormalization group (DMRG) simulations we reveal a dome of large binding energies at around $30\%$ doping, accompanied by a change of the Fermi surface volume and a crossover from extended to tightly bound hole pairs. Our work provides a microscopic theory of pairing in the doped mixD system with dominant repulsion, closely related to bilayer, Ni-based superconductors, and our predictions can be tested in state-of-the-art quantum simulators.

categories: cond-mat.str-el cond-mat.quant-gas cond-mat.supr-con quant-ph


title: Robotic Offline RL from Internet Videos via Value-Function Pre-Training

abstract: Pre-training on Internet data has proven to be a key ingredient for broad generalization in many modern ML systems. What would it take to enable such capabilities in robotic reinforcement learning (RL)? Offline RL methods, which learn from datasets of robot experience, offer one way to leverage prior data into the robotic learning pipeline. However, these methods have a "type mismatch" with video data (such as Ego4D), the largest prior datasets available for robotics, since video offers observation-only experience without the action or reward annotations needed for RL methods. In this paper, we develop a system for leveraging large-scale human video datasets in robotic offline RL, based entirely on learning value functions via temporal-difference learning. We show that value learning on video datasets learns representations that are more conducive to downstream robotic offline RL than other approaches for learning from video data. Our system, called V-PTR, combines the benefits of pre-training on video data with robotic offline RL approaches that train on diverse robot data, resulting in value functions and policies for manipulation tasks that perform better, act robustly, and generalize broadly. On several manipulation tasks on a real WidowX robot, our framework produces policies that greatly improve over prior methods. Our video and additional details can be found at https://dibyaghosh.com/vptr/


title: MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentation

abstract: We present MosaicFusion, a simple yet effective diffusion-based data augmentation approach for large vocabulary instance segmentation. Our method is training-free and does not rely on any label supervision. Two key designs enable us to employ an off-the-shelf text-to-image diffusion model as a useful dataset generator for object instances and mask annotations. First, we divide an image canvas into several regions and perform a single round of diffusion process to generate multiple instances simultaneously, conditioning on different text prompts. Second, we obtain corresponding instance masks by aggregating cross-attention maps associated with object prompts across layers and diffusion time steps, followed by simple thresholding and edge-aware refinement processing. Without bells and whistles, our MosaicFusion can produce a significant amount of synthetic labeled data for both rare and novel categories. Experimental results on the challenging LVIS long-tailed and open-vocabulary benchmarks demonstrate that MosaicFusion can significantly improve the performance of existing instance segmentation models, especially for rare and novel categories. Code will be released at https://github.com/Jiahao000/MosaicFusion.


title: E(2)-Equivariant Graph Planning for Navigation

abstract: Learning for robot navigation presents a critical and challenging task. The scarcity and costliness of real-world datasets necessitate efficient learning approaches. In this letter, we exploit Euclidean symmetry in planning for 2D navigation, which originates from Euclidean transformations between reference frames and enables parameter sharing. To address the challenges of unstructured environments, we formulate the navigation problem as planning on a geometric graph and develop an equivariant message passing network to perform value iteration. Furthermore, to handle multi-camera input, we propose a learnable equivariant layer to lift features to a desired space. We conduct comprehensive evaluations across five diverse tasks encompassing structured and unstructured environments, along with maps of known and unknown, given point goals or semantic goals. Our experiments confirm the substantial benefits on training efficiency, stability, and generalization. More details can be found at the project website: https://lhy.xyz/e2-planning/.


title: What is the Title of this Paper? Solving logic puzzles using algorithms

abstract: This work delves into the realm of logic puzzles by focusing on the Knight and Knave problems popularized by Raymond Smullyan in his book series "What is the Name of This Book?". The puzzles revolve around characters known as Knights (truth-tellers) and Knaves (liars), challenging solvers to determine the true identity of each person based on their statements. This paper explores the utilization of Python algorithms to automate the process of solving these puzzles, offering a computational approach that enhances efficiency and accessibility. In this work, we aim to develop a Python algorithm capable of parsing and analyzing the statements provided in the Knight and Knave puzzles. A logical reasoning framework is integrated within the algorithm to deduce the identities of the characters based on their statements. The algorithm processes the input statements, create a knowledge base, and make deductions following the rules of Knight and Knave logic. The developed algorithm is thoroughly tested on various instances of Knight and Knave puzzles, comparing its results to known solutions and manual approaches. We further expand the scope of the problem by introducing a Normal (who can sometimes lie and sometimes say the truth).


title: A review of blended learning in contrast with traditional learning in Ghanaian universities

abstract: Blended learning methods constructively fuses more than one delivery mode. It makes use of technology to meet the exigency of the times. Covid-19 pandemic has also strengthened the need for teaching and learning to be in modes other than face-to-face interactions. In this research, a state-of-the-art review is conducted on the impact of blended learning in three universities in Ghana. These universities are selected based on a number of reasons including the use of the uniRank ranking system. It has been shown that the Learning management system is implemented in two out of three universities considered in this research.


title: Privacy Preserving Machine Learning for Behavioral Authentication Systems

abstract: A behavioral authentication (BA) system uses the behavioral characteristics of users to verify their identity claims. A BA verification algorithm can be constructed by training a neural network (NN) classifier on users' profiles. The trained NN model classifies the presented verification data, and if the classification matches the claimed identity, the verification algorithm accepts the claim. This classification-based approach removes the need to maintain a profile database. However, similar to other NN architectures, the NN classifier of the BA system is vulnerable to privacy attacks. To protect the privacy of training and test data used in an NN different techniques are widely used. In this paper, our focus is on a non-crypto-based approach, and we used random projection (RP) to ensure data privacy in an NN model. RP is a distance-preserving transformation based on a random matrix. Before sharing the profiles with the verifier, users will transform their profiles by RP and keep their matrices secret. To reduce the computation load in RP, we use sparse random projection, which is very effective for low-compute devices. Along with correctness and security properties, our system can ensure the changeability property of the BA system. We also introduce an ML-based privacy attack, and our proposed system is robust against this and other privacy and security attacks. We implemented our approach on three existing behavioral BA systems and achieved a below 2.0% FRR and a below 1.0% FAR rate. Moreover, the machine learning-based privacy attacker can only recover below 3.0% to 12.0% of features from a portion of the projected profiles. However, these recovered features are not sufficient to know details about the users' behavioral pattern or to be used in a subsequent attack. Our approach is general and can be used in other NN-based BA systems as well as in traditional biometric systems.


title: NeuroCADR: Drug Repurposing to Reveal Novel Anti-Epileptic Drug Candidates Through an Integrated Computational Approach

abstract: Drug repurposing is an emerging approach for drug discovery involving the reassignment of existing drugs for novel purposes. An alternative to the traditional de novo process of drug development, repurposed drugs are faster, cheaper, and less failure prone than drugs developed from traditional methods. Recently, drug repurposing has been performed in silico, in which databases of drugs and chemical information are used to determine interactions between target proteins and drug molecules to identify potential drug candidates. A proposed algorithm is NeuroCADR, a novel system for drug repurposing via a multi-pronged approach consisting of k-nearest neighbor algorithms (KNN), random forest classification, and decision trees. Data was sourced from several databases consisting of interactions between diseases, symptoms, genes, and affiliated drug molecules, which were then compiled into datasets expressed in binary. The proposed method displayed a high level of accuracy, outperforming nearly all in silico approaches. NeuroCADR was performed on epilepsy, a condition characterized by seizures, periods of time with bursts of uncontrolled electrical activity in brain cells. Existing drugs for epilepsy can be ineffective and expensive, revealing a need for new antiepileptic drugs. NeuroCADR identified novel drug candidates for epilepsy that can be further approved through clinical trials. The algorithm has the potential to determine possible drug combinations to prescribe a patient based on a patient's prior medical history. This project examines NeuroCADR, a novel approach to computational drug repurposing capable of revealing potential drug candidates in neurological diseases such as epilepsy.


title: Individual Discrete Logarithm with Sublattice Reduction

abstract: The Number Field Sieve and its numerous variants is the best algorithm to compute discrete logarithms in medium and large characteristic finite fields. When the extension degree n is composite and the characteristic p is of medium size, the Tower variant (TNFS) is asymptotically the most efficient one. Our work deals with the last main step, namely the individual logarithm step, that computes a smooth decomposition of a given target T in the finite field thanks to two distinct phases: an initial splitting step, and a descent tree. In this article, we improve on the current state-of-the-art Guillevic's algorithm dedicated to the initial splitting step for composite n. While still exploiting the proper subfields of the target finite field, we modify the lattice reduction subroutine that creates a lift in a number field of the target T. Our algorithm returns lifted elements with lower degrees and coefficients, resulting in lower norms in the number field. The lifted elements are not only much likely to be smooth because they have smaller norms, but it permits to set a smaller smoothness bound for the descent tree. Asymptotically, our algorithm is faster and works for a larger area of finite fields than Guillevic's algorithm, being now relevant even when the medium characteristic p is such that Lpn (1/3) $\le$ p < Lpn (1/2). In practice, we conduct experiments on 500-bit and 700-bit composite finite fields: Our method becomes more efficient as the largest non trivial divisor of n grows, being thus particularly adapted to even extension degrees.


title: AI-Driven Personalised Offloading Device Prescriptions: A Cutting-Edge Approach to Preventing Diabetes-Related Plantar Forefoot Ulcers and Complications

abstract: Diabetes-related foot ulcers and complications are a significant concern for individuals with diabetes, leading to severe health implications such as lower-limb amputation and reduced quality of life. This chapter discusses applying AI-driven personalised offloading device prescriptions as an advanced solution for preventing such conditions. By harnessing the capabilities of artificial intelligence, this cutting-edge approach enables the prescription of offloading devices tailored to each patient's specific requirements. This includes the patient's preferences on offloading devices such as footwear and foot orthotics and their adaptations that suit the patient's intention of use and lifestyle. Through a series of studies, real-world data analysis and machine learning algorithms, high-risk areas can be identified, facilitating the recommendation of precise offloading strategies, including custom orthotic insoles, shoe adaptations, or specialised footwear. By including patient-specific factors to promote adherence, proactively addressing pressure points and promoting optimal foot mechanics, these personalised offloading devices have the potential to minimise the occurrence of foot ulcers and associated complications. This chapter proposes an AI-powered Clinical Decision Support System (CDSS) to recommend personalised prescriptions of offloading devices (footwear and insoles) for patients with diabetes who are at risk of foot complications. This innovative approach signifies a transformative leap in diabetic foot care, offering promising opportunities for preventive healthcare interventions.


title: Decoding the Alphabet Soup of Degrees in the United States Postsecondary Education System Through Hybrid Method: Database and Text Mining

abstract: This paper proposes a model to predict the levels (e.g., Bachelor, Master, etc.) of postsecondary degree awards that have been ambiguously expressed in the student tracking reports of the National Student Clearinghouse (NSC). The model will be the hybrid of two modules. The first module interprets the relevant abbreviatory elements embedded in NSC reports by referring to a comprehensive database that we have made of nearly 950 abbreviations for degree titles used by American postsecondary educators. The second module is a combination of feature classification and text mining modeled with CNN-BiLSTM, which is preceded by several steps of heavy pre-processing. The model proposed in this paper was trained with four multi-label datasets of different grades of resolution and returned 97.83\% accuracy with the most sophisticated dataset. Such a thorough classification of degree levels will provide insights into the modeling patterns of student success and mobility. To date, such a classification strategy has not been attempted except using manual methods and simple text parsing logic.


title: A Contextual Topic Modeling and Content Analysis of Iranian laws and Regulations

abstract: A constitution is the highest legal document of a country and serves as a guide for the establishment of other laws. The constitution defines the political principles, structure, hierarchy, position, and limits of the political power of a country's government. It determines and guarantees the rights of citizens. This study aimed at topic modeling of Iranian laws. As part of this research, 11760 laws were collected from the Dotic website. Then, topic modeling was conducted on the title and content of the regularizations using LDA. Data analysis with topic modeling led to the identification of 10 topics including Economic, Customs, Housing and Urban Development, Agriculture, Insurance, Legal and judicial, Cultural, Information Technology, Political, and Government. The largest topic, Economic, accounts for 29% of regulations, while the smallest are Political and Government, accounting for 2%. This research utilizes a topic modeling method in exploring law texts and identifying trends in regularizations from 2016-2023. In this study, it was found that regularizations constitute a significant percentage of law, most of which are related to economics and customs. Cultural regularizations have increased in 2023. It can be concluded any law enacted each year can reflect society's conditions and legislators' top concerns.


title: Students Success Modeling: Most Important Factors

abstract: The importance of retention rate for higher education institutions has encouraged data analysts to present various methods to predict at-risk students. The present study, motivated by the same encouragement, proposes a deep learning model trained with 121 features of diverse categories extracted or engineered out of the records of 60,822 postsecondary students. The model undertakes to identify students likely to graduate, the ones likely to transfer to a different school, and the ones likely to drop out and leave their higher education unfinished. This study undertakes to adjust its predictive methods for different stages of curricular progress of students. The temporal aspects introduced for this purpose are accounted for by incorporating layers of LSTM in the model. Our experiments demonstrate that distinguishing between to-be-graduate and at-risk students is reasonably achievable in the earliest stages, and then it rapidly improves, but the resolution within the latter category (dropout vs. transfer) depends on data accumulated over time. However, the model remarkably foresees the fate of students who stay in the school for three years. The model is also assigned to present the weightiest features in the procedure of prediction, both on institutional and student levels. A large, diverse sample size along with the investigation of more than one hundred extracted or engineered features in our study provide new insights into variables that affect students success, predict dropouts with reasonable accuracy, and shed light on the less investigated issue of transfer between colleges. More importantly, by providing individual-level predictions (as opposed to school-level predictions) and addressing the outcomes of transfers, this study improves the use of ML in the prediction of educational outcomes.


title: Using Curriculum Theory to Inform Approaches to Generative AI in Schools

abstract: In an educational landscape dramatically altered by the swift proliferation of Large Language Models, this essay interrogates the urgent this essay interrogates the urgent pedagogical modifications required in secondary schooling. Anchored in Madeline Grumet's triadic framework of curriculum inquiry, the study delineates the multifaceted relationship between Generative AI and Elliot Eisner's explicit, implicit, and null curriculum concepts. It scrutinizes the logistical and ethical challenges, such as the reliability of AI detectors, that educators confront when attempting to assimilate this nascent technology into long-standing curricular structures. Engaging with Ted Aoki's theory of the "zone of between", the essay illuminates educators' dilemmas in reconciling prescriptive curricular aims with the fluid realities of classroom life, all within an educational milieu in constant flux due to Generative AI. The paper culminates in a reflective analysis by the researcher, identifying avenues for further scholarly investigation within each of Grumet's constitutive strands of curriculum theory, thereby providing a roadmap for future research on Generative AI's transformative impact on educational practice.


title: Data Commons

abstract: Publicly available data from open sources (e.g., United States Census Bureau (Census), World Health Organization (WHO), Intergovernmental Panel on Climate Change (IPCC)) are vital resources for policy makers, students and researchers across different disciplines. Combining data from different sources requires the user to reconcile the differences in schemas, formats, assumptions, and more. This data wrangling is time consuming, tedious and needs to be repeated by every user of the data. Our goal with Data Commons (DC) is to help make public data accessible and useful to those who want to understand this data and use it to solve societal challenges and opportunities. We do the data processing and make the processed data widely available via standard schemas and Cloud APIs. Data Commons is a distributed network of sites that publish data in a common schema and interoperate using the Data Commons APIs. Data from different Data Commons can be joined easily. The aggregate of these Data Commons can be viewed as a single Knowledge Graph. This Knowledge Graph can then be searched over using Natural Language questions utilizing advances in Large Language Models. This paper describes the architecture of Data Commons, some of the major deployments and highlights directions for future work.


title: Originality and the Future of Copyright in an Age of Generative AI

abstract: This papers explores the question of human authorship when works are created with generative AI tools.


title: Book Chapter in Computational Demography and Health

abstract: Recent developments in computing, data entry and generation, and analytic tools have changed the landscape of modern demography and health research. These changes have come to be known as computational demography, big data, and precision health in the field. This emerging interdisciplinary research comprises social scientists, physical scientists, engineers, data scientists, and disease experts. This work has changed how we use administrative data, conduct surveys, and allow for complex behavioral studies via big data (electronic trace data from mobile phones, apps, etc.). This chapter reviews this emerging field's new data sources, methods, and applications.


title: The Return on Investment in AI Ethics: A Holistic Framework

abstract: We propose a Holistic Return on Ethics (HROE) framework for understanding the return on organizational investments in artificial intelligence (AI) ethics efforts. This framework is useful for organizations that wish to quantify the return for their investment decisions. The framework identifies the direct economic returns of such investments, the indirect paths to return through intangibles associated with organizational reputation, and real options associated with capabilities. The holistic framework ultimately provides organizations with the competency to employ and justify AI ethics investments.


title: Mathematical Modeling and Optimal Control of Untrue Information : Dynamic SEIZ in Online Social Networks

abstract: We propose to model the phenomenon of the spread of a rumor in this paper. We manipulate a model that is based on SEIR model that specializes in spreading rumors. In the second part, we introduce a control strategy to fight against the diffusion of the rumor. Our main objective is to characterize the three optimal controls that minimize the number of spreaders, susceptibles who enter and spread the rumor, and skeptics. For that matter, using the maximum principle of Pontryagin, we prove the existence and give characterization of our controls. To illustrate the theoretical results obtained, numerical simulations are given to concretize our approach.


title: Beyond Traditional Teaching: The Potential of Large Language Models and Chatbots in Graduate Engineering Education

abstract: In the rapidly evolving landscape of education, digital technologies have repeatedly disrupted traditional pedagogical methods. This paper explores the latest of these disruptions: the potential integration of large language models (LLMs) and chatbots into graduate engineering education. We begin by tracing historical and technological disruptions to provide context and then introduce key terms such as machine learning and deep learning and the underlying mechanisms of recent advancements, namely attention/transformer models and graphics processing units. The heart of our investigation lies in the application of an LLM-based chatbot in a graduate fluid mechanics course. We developed a question bank from the course material and assessed the chatbot's ability to provide accurate, insightful responses. The results are encouraging, demonstrating not only the bot's ability to effectively answer complex questions but also the potential advantages of chatbot usage in the classroom, such as the promotion of self-paced learning, the provision of instantaneous feedback, and the reduction of instructors' workload. The study also examines the transformative effect of intelligent prompting on enhancing the chatbot's performance. Furthermore, we demonstrate how powerful plugins like Wolfram Alpha for mathematical problem-solving and code interpretation can significantly extend the chatbot's capabilities, transforming it into a comprehensive educational tool. While acknowledging the challenges and ethical implications surrounding the use of such AI models in education, we advocate for a balanced approach. The use of LLMs and chatbots in graduate education can be greatly beneficial but requires ongoing evaluation and adaptation to ensure ethical and efficient use.


title: Implementing Learning Principles with a Personal AI Tutor: A Case Study

abstract: Effective learning strategies based on principles like personalization, retrieval practice, and spaced repetition are often challenging to implement due to practical constraints. Here we explore the integration of AI tutors to complement learning programs in accordance with learning sciences. A semester-long study was conducted at UniDistance Suisse, where an AI tutor app was provided to psychology students taking a neuroscience course (N=51). After automatically generating microlearning questions from existing course materials using GPT-3, the AI tutor developed a dynamic neural-network model of each student's grasp of key concepts. This enabled the implementation of distributed retrieval practice, personalized to each student's individual level and abilities. The results indicate that students who actively engaged with the AI tutor achieved significantly higher grades. Moreover, active engagement led to an average improvement of up to 15 percentile points compared to a parallel course without AI tutor. Additionally, the grasp strongly correlated with the exam grade, thus validating the relevance of neural-network predictions. This research demonstrates the ability of personal AI tutors to model human learning processes and effectively enhance academic performance. By integrating AI tutors into their programs, educators can offer students personalized learning experiences grounded in the principles of learning sciences, thereby addressing the challenges associated with implementing effective learning strategies. These findings contribute to the growing body of knowledge on the transformative potential of AI in education.


title: Applying BioBERT to Extract Germline Gene-Disease Associations for Building a Knowledge Graph from the Biomedical Literature

abstract: Published biomedical information has and continues to rapidly increase. The recent advancements in Natural Language Processing (NLP), have generated considerable interest in automating the extraction, normalization, and representation of biomedical knowledge about entities such as genes and diseases. Our study analyzes germline abstracts in the construction of knowledge graphs of the of the immense work that has been done in this area for genes and diseases. This paper presents SimpleGermKG, an automatic knowledge graph construction approach that connects germline genes and diseases. For the extraction of genes and diseases, we employ BioBERT, a pre-trained BERT model on biomedical corpora. We propose an ontology-based and rule-based algorithm to standardize and disambiguate medical terms. For semantic relationships between articles, genes, and diseases, we implemented a part-whole relation approach to connect each entity with its data source and visualize them in a graph-based knowledge representation. Lastly, we discuss the knowledge graph applications, limitations, and challenges to inspire the future research of germline corpora. Our knowledge graph contains 297 genes, 130 diseases, and 46,747 triples. Graph-based visualizations are used to show the results.


title: Contraction map sets with an external factor and weakly fixed points

abstract: In this paper I introduce the property CD which is a more convenient variant of the UC property and show one of the possible relationships between them, I also extend the concept of a fixed point, introducing the concept of a weak fixation of a point about a sequence. I introduce contraction map sets with an external factor and formulate a theorem for them, on which the main focus of this article falls.


title: Using Large Language Models to Generate, Validate, and Apply User Intent Taxonomies

abstract: Log data can reveal valuable information about how users interact with Web search services, what they want, and how satisfied they are. However, analyzing user intents in log data is not easy, especially for emerging forms of Web search such as AI-driven chat. To understand user intents from log data, we need a way to label them with meaningful categories that capture their diversity and dynamics. Existing methods rely on manual or machine-learned labeling, which are either expensive or inflexible for large and dynamic datasets. We propose a novel solution using large language models (LLMs), which can generate rich and relevant concepts, descriptions, and examples for user intents. However, using LLMs to generate a user intent taxonomy and apply it for log analysis can be problematic for two main reasons: (1) such a taxonomy is not externally validated; and (2) there may be an undesirable feedback loop. To address this, we propose a new methodology with human experts and assessors to verify the quality of the LLM-generated taxonomy. We also present an end-to-end pipeline that uses an LLM with human-in-the-loop to produce, refine, and apply labels for user intent analysis in log data. We demonstrate its effectiveness by uncovering new insights into user intents from search and chat logs from the Microsoft Bing commercial search engine. The proposed work's novelty stems from the method for generating purpose-driven user intent taxonomies with strong validation. This method not only helps remove methodological and practical bottlenecks from intent-focused research, but also provides a new framework for generating, validating, and applying other kinds of taxonomies in a scalable and adaptable way with reasonable human effort.


title: InvestLM: A Large Language Model for Investment using Financial Domain Instruction Tuning

abstract: We present a new financial domain large language model, InvestLM, tuned on LLaMA-65B (Touvron et al., 2023), using a carefully curated instruction dataset related to financial investment. Inspired by less-is-more-for-alignment (Zhou et al., 2023), we manually curate a small yet diverse instruction dataset, covering a wide range of financial related topics, from Chartered Financial Analyst (CFA) exam questions to SEC filings to Stackexchange quantitative finance discussions. InvestLM shows strong capabilities in understanding financial text and provides helpful responses to investment related questions. Financial experts, including hedge fund managers and research analysts, rate InvestLM's response as comparable to those of state-of-the-art commercial models (GPT-3.5, GPT-4 and Claude-2). Zero-shot evaluation on a set of financial NLP benchmarks demonstrates strong generalizability. From a research perspective, this work suggests that a high-quality domain specific LLM can be tuned using a small set of carefully curated instructions on a well-trained foundation model, which is consistent with the Superficial Alignment Hypothesis (Zhou et al., 2023). From a practical perspective, this work develops a state-of-the-art financial domain LLM with superior capability in understanding financial texts and providing helpful investment advice, potentially enhancing the work efficiency of financial professionals. We release the model parameters to the research community.


title: Personality Profiling: How informative are social media profiles in predicting personal information?

abstract: Personality profiling has been utilised by companies for targeted advertising, political campaigns and vaccine campaigns. However, the accuracy and versatility of such models still remains relatively unknown. Consequently, we aim to explore the extent to which peoples' online digital footprints can be used to profile their Myers-Briggs personality type. We analyse and compare the results of four models: logistic regression, naive Bayes, support vector machines (SVMs) and random forests. We discover that a SVM model achieves the best accuracy of 20.95% for predicting someones complete personality type. However, logistic regression models perform only marginally worse and are significantly faster to train and perform predictions. We discover that many labelled datasets present substantial class imbalances of personal characteristics on social media, including our own. As a result, we highlight the need for attentive consideration when reporting model performance on these datasets and compare a number of methods for fixing the class-imbalance problems. Moreover, we develop a statistical framework for assessing the importance of different sets of features in our models. We discover some features to be more informative than others in the Intuitive/Sensory (p = 0.032) and Thinking/Feeling (p = 0.019) models. While we apply these methods to Myers-Briggs personality profiling, they could be more generally used for any labelling of individuals on social media.


title: Causal Discovery and Counterfactual Explanations for Personalized Student Learning

abstract: The paper focuses on identifying the causes of student performance to provide personalized recommendations for improving pass rates. We introduce the need to move beyond predictive models and instead identify causal relationships. We propose using causal discovery techniques to achieve this. The study's main contributions include using causal discovery to identify causal predictors of student performance and applying counterfactual analysis to provide personalized recommendations. The paper describes the application of causal discovery methods, specifically the PC algorithm, to real-life student performance data. It addresses challenges such as sample size limitations and emphasizes the role of domain knowledge in causal discovery. The results reveal the identified causal relationships, such as the influence of earlier test grades and mathematical ability on final student performance. Limitations of this study include the reliance on domain expertise for accurate causal discovery, and the necessity of larger sample sizes for reliable results. The potential for incorrect causal structure estimations is acknowledged. A major challenge remains, which is the real-time implementation and validation of counterfactual recommendations. In conclusion, the paper demonstrates the value of causal discovery for understanding student performance and providing personalized recommendations. It highlights the challenges, benefits, and limitations of using causal inference in an educational context, setting the stage for future studies to further explore and refine these methods.


title: On the solutions of $\varphi(dn) = \varphi(d(n+h))$

abstract: This is a short note about a chapter in the author's bachelor thesis regarding a paper by Ford concerning a conjecture by Erd\H{o}s.