title: Tighter Generalisation Bounds via Interpolation

abstract: This paper contains a recipe for deriving new PAC-Bayes generalisation bounds based on the $(f, \Gamma)$-divergence, and, in addition, presents PAC-Bayes generalisation bounds where we interpolate between a series of probability divergences (including but not limited to KL, Wasserstein, and total variation), making the best out of many worlds depending on the posterior distributions properties. We explore the tightness of these bounds and connect them to earlier results from statistical learning, which are specific cases. We also instantiate our bounds as training objectives, yielding non-trivial guarantees and practical performances.


title: You Can REST Now: Automated Specification Inference and Black-Box Testing of RESTful APIs with Large Language Models

abstract: RESTful APIs are popular web services, requiring documentation to ease their comprehension, reusability and testing practices. The OpenAPI Specification (OAS) is a widely adopted and machine-readable format used to document such APIs. However, manually documenting RESTful APIs is a time-consuming and error-prone task, resulting in unavailable, incomplete, or imprecise documentation. As RESTful API testing tools require an OpenAPI specification as input, insufficient or informal documentation hampers testing quality. Recently, Large Language Models (LLMs) have demonstrated exceptional abilities to automate tasks based on their colossal training data. Accordingly, such capabilities could be utilized to assist the documentation and testing process of RESTful APIs. In this paper, we present RESTSpecIT, the first automated RESTful API specification inference and black-box testing approach leveraging LLMs. The approach requires minimal user input compared to state-of-the-art RESTful API inference and testing tools; Given an API name and an LLM key, HTTP requests are generated and mutated with data returned by the LLM. By sending the requests to the API endpoint, HTTP responses can be analyzed for inference and testing purposes. RESTSpecIT utilizes an in-context prompt masking strategy, requiring no model fine-tuning. Our evaluation demonstrates that RESTSpecIT is capable of: (1) inferring specifications with 85.05% of GET routes and 81.05% of query parameters found on average, (2) discovering undocumented and valid routes and parameters, and (3) uncovering server errors in RESTful APIs. Inferred specifications can also be used as testing tool inputs.


title: Hennings TQFTs for Cobordisms Decorated With Cohomology Classes

abstract: Starting from an abelian group $G$ and a factorizable ribbon Hopf $G$-bialgebra $H$, we construct a TQFT $J_H$ for connected framed cobordisms between connected surfaces with connected boundary decorated with cohomology classes with coefficients in $G$. When restricted to the subcategory of cobordisms with trivial decorations, our functor recovers a special case of Kerler-Lyubashenko TQFTs, namely those associated with factorizable ribbon Hopf algebras. Our result is inspired by the work of Blanchet-Costantino-Geer-Patureau, who constructed non-semisimple TQFTs for admissible decorated cobordisms using the unrolled quantum group of $\mathfrak{sl}_2$, and by that of Geer-Ha-Patureau, who reformulated the underlying invariants of admissible decorated $3$-manifolds using ribbon Hopf $G$-coalgebras. Our work represents the first step towards a homological model for non-semisimple TQFTs decorated with cohomology classes that appears in a conjecture by the first two authors.


title: Millimeter-scale freestanding superconducting infinite-layer nickelate membranes

abstract: Progress in the study of infinite-layer nickelates has always been highly linked to materials advances. In particular, the recent development of superconductivity via hole-doping was predicated on the controlled synthesis of Ni in a very high oxidation state, and subsequent topotactic reduction to a very low oxidation state, currently limited to epitaxial thin films. Here we demonstrate a process to combine these steps with a heterostructure which includes an epitaxial soluble buffer layer, enabling the release of freestanding membranes of (Nd,Sr)NiO2 encapsulated in SrTiO3, which serves as a protective layer. The membranes have comparable structural and electronic properties to that of optimized thin films, and range in lateral dimensions from millimeters to ~100 micron fragments, depending on the degree of strain released with respect to the initial substrate. The changes in the superconducting transition temperature associated with membrane release are quite similar to those reported for substrate and pressure variations, suggestive of a common underlying mechanism. These membranes structures should provide a versatile platform for a range of experimental studies and devices free from substrate constraints.

categories: cond-mat.mtrl-sci cond-mat.mes-hall cond-mat.str-el cond-mat.supr-con


title: On gauge transformations in twistless torsional Newton--Cartan geometry

abstract: We observe that in type I twistless torsional Newton--Cartan (TTNC) geometry, one can always find (at least locally) a gauge transformation that transforms a specific locally Galilei-invariant function -- that we dub the `locally Galilei-invariant potential' -- to zero, due to the corresponding equation for the gauge parameter taking the form of a Hamilton--Jacobi equation. In the case of type II TTNC geometry, the same gauge fixing may locally be performed by subleading spatial diffeomorphisms. We show (a) how this generalises a classical result in standard Newton--Cartan geometry, and (b) how it allows to parametrise the metric structure of a Galilei manifold as well as the gauge equivalence class of the Bargmann form of TTNC geometry in terms of just the space metric and a unit timelike vector field.


title: Image captioning for Brazilian Portuguese using GRIT model

abstract: This work presents the early development of a model of image captioning for the Brazilian Portuguese language. We used the GRIT (Grid - and Region-based Image captioning Transformer) model to accomplish this work. GRIT is a Transformer-only neural architecture that effectively utilizes two visual features to generate better captions. The GRIT method emerged as a proposal to be a more efficient way to generate image captioning. In this work, we adapt the GRIT model to be trained in a Brazilian Portuguese dataset to have an image captioning method for the Brazilian Portuguese Language.


title: Nature of the diffuse emission sources in the H I supershell in the galaxy IC 1613

abstract: We present a study of the nearby low-metallicity dwarf galaxy IC 1613, focusing on the search for massive stars and related feedback processes, as well as for faint supernova remnants (SNR) in late stages of evolution. We obtained the deepest images of IC 1613 in the narrow-band H{\alpha}, He II and [S II] emission lines and new long-slit spectroscopy observations using several facilities (6-m BTA, 2.5m SAI MSU, and 150RTT telescopes), in combination with the multi-wavelength archival data from MUSE/VLT, VLA, XMM-Newton, and Swift/XRT. Our deep narrow-band photometry identifies several faint shells in the galaxy, and we further investigate their physical characteristics with the new long-slit spectroscopy observations and the archival multi-wavelength data. Based on energy balance calculations and assumptions about their possible nature, we propose that one of the shells is a possible remnant of a supernova explosion. We study five out of eight Wolf-Rayet (WR) star candidates previously published for this galaxy using the He ii emission line mapping, MUSE/VLT archival spectra, and new long-slit spectra. Our analysis discards the considered WR candidates and finds no new ones. We found P Cyg profiles in H{\alpha} line in two stars, which we classify as Luminous Blue Variable (LBV) star candidates. Overall, the galaxy IC 1613 may have a lower rate of WR star formation than previously suggested.


title: A topological algorithm for the Fourier transform of Stokes data at infinity

abstract: We reinterpret a result of T. Mochizuki about the Fourier transform of Stokes data of irregular connections on the Riemann sphere in the language of Stokes local systems due to P. Boalch. We thus obtain a clean topological description of the Stokes matrices of the Fourier transform from infinity to infinity in a large number of cases of one level. In particular, this induces explicit isomorphisms between wild character varieties, in a much larger range of examples than those for which such isomorphisms have previously been written down. We conjecture that these isomorphisms are compatible with the quasi-Hamiltonian structure of the wild character varieties.


title: Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding

abstract: To combat the memory bandwidth-bound nature of autoregressive LLM inference, previous research has proposed the speculative decoding framework. To perform speculative decoding, a small draft model proposes candidate continuations of the input sequence, that are then verified in parallel by the base model. One way to specify the draft model, as used in the recent Medusa decoding framework, is as a collection of light-weight heads, called draft heads, that operate on the base model's hidden states. To date, all existing draft heads have been sequentially independent, meaning that they speculate tokens in the candidate continuation independently of any preceding tokens in the candidate continuation. In this work, we propose Hydra heads, a sequentially dependent, drop-in replacement for standard draft heads that significantly improves speculation accuracy. Decoding with Hydra heads improves throughput compared to Medusa decoding with standard draft heads. We further explore the design space of Hydra head training objectives and architectures, and propose a carefully-tuned Hydra head recipe, which we call Hydra++, that improves decoding throughput by 1.31x and 2.71x compared to Medusa decoding and autoregressive decoding, respectively. Overall, Hydra heads are a simple intervention on standard draft heads that significantly improve the end-to-end speed of draft head based speculative decoding.


title: Opening the AI black box: program synthesis via mechanistic interpretability

abstract: We present MIPS, a novel method for program synthesis based on automated mechanistic interpretability of neural networks trained to perform the desired task, auto-distilling the learned algorithm into Python code. We test MIPS on a benchmark of 62 algorithmic tasks that can be learned by an RNN and find it highly complementary to GPT-4: MIPS solves 32 of them, including 13 that are not solved by GPT-4 (which also solves 30). MIPS uses an integer autoencoder to convert the RNN into a finite state machine, then applies Boolean or integer symbolic regression to capture the learned algorithm. As opposed to large language models, this program synthesis technique makes no use of (and is therefore not limited by) human training data such as algorithms and code from GitHub. We discuss opportunities and challenges for scaling up this approach to make machine-learned models more interpretable and trustworthy.


title: Edu-ConvoKit: An Open-Source Library for Education Conversation Data

abstract: We introduce Edu-ConvoKit, an open-source library designed to handle pre-processing, annotation and analysis of conversation data in education. Resources for analyzing education conversation data are scarce, making the research challenging to perform and therefore hard to access. We address these challenges with Edu-ConvoKit. Edu-ConvoKit is open-source (https://github.com/stanfordnlp/edu-convokit ), pip-installable (https://pypi.org/project/edu-convokit/ ), with comprehensive documentation (https://edu-convokit.readthedocs.io/en/latest/ ). Our demo video is available at: https://youtu.be/zdcI839vAko?si=h9qlnl76ucSuXb8- . We include additional resources, such as Colab applications of Edu-ConvoKit to three diverse education datasets and a repository of Edu-ConvoKit related papers, that can be found in our GitHub repository.


title: Vortex spin liquid with fractional quantum spin Hall effect in moir\'e Chern bands

abstract: Integer and fractional quantum anomalous Hall (QAH) effects have been widely seen in moir\'e systems. Recently there is even observation of a time reversal invariant fractional quantum spin hall (FQSH) state at filling $n=3$ in twisted MoTe$_2$ bilayer. We consider a pair of half-filled $C=\pm 1$ Chern band in the two valleys, similar to the well-studied quantum Hall bilayer, but now with opposite chiralities. Due to the strong inter-valley repulsion, we expect a charge gap opening with low energy physics dominated by the neutral inter-valley excitons. However, the presence of an effective `flux' frustrates exciton condensation by proliferating vortices. Here we construct a vortex liquid of excitons dubbed as vortex spin liquid (VSL), from exciton pairing of the composite fermions in the decoupled composite Fermi liquids (CFL) phase. This insulator is a quantum spin liquid with gapless spin excitations carried by the flux of an emergent U(1) gauge field. Additionally, there exist neutral and spinless Fermi surfaces formed by fermionic vortices of a nearby inter-valley-coherent (IVC) order. Unlike a conventional Mott insulator, the VSL phase also exhibits FQSH effect with gapless helical charge modes along the edge. Our work suggests a new platform to search for quantum spin liquid enriched by fractional quantum spin Hall effect. We also point out the possibility of quantum oscillations and thermal Hall effect under Zeeman field in this exotic insulator.


title: Portfolio Time Consistency and Utility Weighted Discount Rates

abstract: Merton portfolio management problem is studied in this paper within a stochastic volatility, non constant time discount rate, and power utility framework. This problem is time inconsistent and the way out of this predicament is to consider the subgame perfect strategies. The later are characterized through an extended Hamilton Jacobi Bellman (HJB) equation. A fixed point iteration is employed to solve the extended HJB equation. This is done in a two stage approach: in a first step the utility weighted discount rate is introduced and characterized as the fixed point of a certain operator; in the second step the value function is determined through a linear parabolic partial differential equation. Numerical experiments explore the effect of the time discount rate on the subgame perfect and precommitment strategies.


title: A Light-weight and Unsupervised Method for Near Real-time Behavioral Analysis using Operational Data Measurement

abstract: Monitoring the status of large computing systems is essential to identify unexpected behavior and improve their performance and uptime. However, due to the large-scale and distributed design of such computing systems as well as a large number of monitoring parameters, automated monitoring methods should be applied. Such automatic monitoring methods should also have the ability to adapt themselves to the continuous changes in the computing system. In addition, they should be able to identify behavioral anomalies in useful time, to perform appropriate reactions. This work proposes a general lightweight and unsupervised method for near real-time anomaly detection using operational data measurement on large computing systems. The proposed model requires as little as 4 hours of data and 50 epochs for each training process to accurately resemble the behavioral pattern of computing systems.


title: Unsupervised Motion Retargeting for Human-Robot Imitation

abstract: This early-stage research work aims to improve online human-robot imitation by translating sequences of joint positions from the domain of human motions to a domain of motions achievable by a given robot, thus constrained by its embodiment. Leveraging the generalization capabilities of deep learning methods, we address this problem by proposing an encoder-decoder neural network model performing domain-to-domain translation. In order to train such a model, one could use pairs of associated robot and human motions. Though, such paired data is extremely rare in practice, and tedious to collect. Therefore, we turn towards deep learning methods for unpaired domain-to-domain translation, that we adapt in order to perform human-robot imitation.


title: Quantifying Similarity: Text-Mining Approaches to Evaluate ChatGPT and Google Bard Content in Relation to BioMedical Literature

abstract: Background: The emergence of generative AI tools, empowered by Large Language Models (LLMs), has shown powerful capabilities in generating content. To date, the assessment of the usefulness of such content, generated by what is known as prompt engineering, has become an interesting research question. Objectives Using the mean of prompt engineering, we assess the similarity and closeness of such contents to real literature produced by scientists. Methods In this exploratory analysis, (1) we prompt-engineer ChatGPT and Google Bard to generate clinical content to be compared with literature counterparts, (2) we assess the similarities of the contents generated by comparing them with counterparts from biomedical literature. Our approach is to use text-mining approaches to compare documents and associated bigrams and to use network analysis to assess the terms' centrality. Results The experiments demonstrated that ChatGPT outperformed Google Bard in cosine document similarity (38% to 34%), Jaccard document similarity (23% to 19%), TF-IDF bigram similarity (47% to 41%), and term network centrality (degree and closeness). We also found new links that emerged in ChatGPT bigram networks that did not exist in literature bigram networks. Conclusions: The obtained similarity results show that ChatGPT outperformed Google Bard in document similarity, bigrams, and degree and closeness centrality. We also observed that ChatGPT offers linkage to terms that are connected in the literature. Such connections could inspire asking interesting questions and generate new hypotheses.


title: On the detection of alpha emission from a low-voltage DC deuterium discharge with palladium electrodes

abstract: The possibility of nuclear reactions within a low voltage DC deuterium discharge with palladium electrodes is addressed. The solid-state nuclear track detector known as CR-39 was chosen to investigate the emission of energetic charged particles from the electrodes. A partially automated imaging platform and feature classification process was developed to scan the CR-39 surface and detect tracks. Typical discharge parameters were 10 Torr deuterium, 5-7 mm electrode gap distance, 20-40 mA/cm$^2$ current density, and -500 $\pm$ 100 V cathode bias. After discharge treatments to varied ion-cathode fluences, tracks formed in CR-39 which consistently corresponded to 138 $\pm$ 21 keV alpha particles emitted from the palladium electrodes. The track densities for deuterium discharges were often $\sim$100 times above controls with hydrogen and helium. Currently, there are no known mechanisms to accelerate ions to these energies within the apparatus. The production of energetic alpha particles with no source of helium or a means to accelerate the ions to such high energies indicate a nuclear origin. From particle trajectory estimates based on track geometries, it was concluded the reactions originated at the Pd electrodes and not from external sources such as atmospheric radon or cosmic rays.


title: Enhancing Accessibility of Rural Populations through Vehicle-based Services

abstract: Improving access to essential public services like healthcare and education is crucial for human development, particularly in rural Sub-Saharan Africa. However, limited reliable transportation and sparse public facilities present significant challenges. Mobile facilities like mobile clinics offer a cost-effective solution to enhance spatial accessibility for the rural population.Public authorities require detailed demand distribution data to allocate resources efficiently and maximize the impact of mobile facilities. This includes determining optimal vehicle service stop locations and estimating operational costs. Our integrated approach utilizes GIS data and an accessibility scaling factor to assess spatial accessibility for rural populations. We tailor demand structures to account for remote and underserved populations. To reduce average travel distances to 5 km, we apply a clustering algorithm and optimize vehicle service stop locations. In a case study in rural Ethiopia, focusing on four key public services, our analysis demonstrates that mobile facilities can address 39-62\% of unmet demand, even in areas with widely dispersed populations. This approach aids decision-makers, including fleet operators, policymakers, and public authorities in Sub-Saharan Africa, during project evaluation and planning for mobile facilities. By enhancing spatial accessibility and optimizing resource allocation, our methodology contributes to the effective delivery of essential public services to underserved populations.


title: A Closer Look at the Limitations of Instruction Tuning

abstract: Instruction Tuning (IT), the process of training large language models (LLMs) using instruction-response pairs, has emerged as the predominant method for transforming base pre-trained LLMs into open-domain conversational agents. While IT has achieved notable success and widespread adoption, its limitations and shortcomings remain underexplored. In this paper, through rigorous experiments and an in-depth analysis of the changes LLMs undergo through IT, we reveal various limitations of IT. In particular, we show that (1) IT fails to enhance knowledge or skills in LLMs. LoRA fine-tuning is limited to learning response initiation and style tokens, and full-parameter fine-tuning leads to knowledge degradation. (2) Copying response patterns from IT datasets derived from knowledgeable sources leads to a decline in response quality. (3) Full-parameter fine-tuning increases hallucination by inaccurately borrowing tokens from conceptually similar instances in the IT dataset for generating responses. (4) Popular methods to improve IT do not lead to performance improvements over a simple LoRA fine-tuned model. Our findings reveal that responses generated solely from pre-trained knowledge consistently outperform responses by models that learn any form of new knowledge from IT on open-source datasets. We hope the insights and challenges revealed in this paper inspire future work in related directions.


title: More Agents Is All You Need

abstract: We find that, simply via a sampling-and-voting method, the performance of large language models (LLMs) scales with the number of agents instantiated. Also, this method is orthogonal to existing complicated methods to further enhance LLMs, while the degree of enhancement is correlated to the task difficulty. We conduct comprehensive experiments on a wide range of LLM benchmarks to verify the presence of our finding, and to study the properties that can facilitate its occurrence. Our code is publicly available at: \url{https://anonymous.4open.science/r/more_agent_is_all_you_need}.


title: Large Language Model for Table Processing: A Survey

abstract: Tables, typically two-dimensional and structured to store large amounts of data, are essential in daily activities like database queries, spreadsheet calculations, and generating reports from web tables. Automating these table-centric tasks with Large Language Models (LLMs) offers significant public benefits, garnering interest from academia and industry. This survey provides an extensive overview of table tasks, encompassing not only the traditional areas like table question answering (Table QA) and fact verification, but also newly emphasized aspects such as table manipulation and advanced table data analysis. Additionally, it goes beyond the early strategies of pre-training and fine-tuning small language models, to include recent paradigms in LLM usage. The focus here is particularly on instruction-tuning, prompting, and agent-based approaches within the realm of LLMs. Finally, we highlight several challenges, ranging from private deployment and efficient inference to the development of extensive benchmarks for table manipulation and advanced data analysis.


title: History of generative Artificial Intelligence (AI) chatbots: past, present, and future development

abstract: This research provides an in-depth comprehensive review of the progress of chatbot technology over time, from the initial basic systems relying on rules to today's advanced conversational bots powered by artificial intelligence. Spanning many decades, the paper explores the major milestones, innovations, and paradigm shifts that have driven the evolution of chatbots. Looking back at the very basic statistical model in 1906 via the early chatbots, such as ELIZA and ALICE in the 1960s and 1970s, the study traces key innovations leading to today's advanced conversational agents, such as ChatGPT and Google Bard. The study synthesizes insights from academic literature and industry sources to highlight crucial milestones, including the introduction of Turing tests, influential projects such as CALO, and recent transformer-based models. Tracing the path forward, the paper highlights how natural language processing and machine learning have been integrated into modern chatbots for more sophisticated capabilities. This chronological survey of the chatbot landscape provides a holistic reference to understand the technological and historical factors propelling conversational AI. By synthesizing learnings from this historical analysis, the research offers important context about the developmental trajectory of chatbots and their immense future potential across various field of application which could be the potential take ways for the respective research community and stakeholders.


title: A Survey on Data Selection for LLM Instruction Tuning

abstract: Instruction tuning is a vital step of training large language models (LLM), so how to enhance the effect of instruction tuning has received increased attention. Existing works indicate that the quality of the dataset is more crucial than the quantity during instruction tuning of LLM. Therefore, recently a lot of studies focus on exploring the methods of selecting high-quality subset from instruction datasets, aiming to reduce training costs and enhance the instruction-following capabilities of LLMs. This paper presents a comprehensive survey on data selection for LLM instruction tuning. Firstly, we introduce the wildly used instruction datasets. Then, we propose a new taxonomy of the data selection methods and provide a detailed introduction of recent advances,and the evaluation strategies and results of data selection methods are also elaborated in detail. Finally, we emphasize the open challenges and present new frontiers of this task.


title: Kink-pair mechanism and low temperature flow-stress behaviour of strontium titanate single crystals

abstract: The mechanical behaviour of strontium titanate exhibits a remarkable behaviour at low temperature, in the so called regime A, where the flow stress experiences two different temperature dependences separated by a noticeably abrupt drop in between. The dislocation microstructure was investigated and, by making adequate use of the kink-pair model, we interpret this behaviour as a transition from the short- to the long-segment limit of kink-pair formation. The fit parameters are found to be physically sound.


title: Zero-Shot Clinical Trial Patient Matching with LLMs

abstract: Matching patients to clinical trials is a key unsolved challenge in bringing new drugs to market. Today, identifying patients who meet a trial's eligibility criteria is highly manual, taking up to 1 hour per patient. Automated screening is challenging, however, as it requires understanding unstructured clinical text. Large language models (LLMs) offer a promising solution. In this work, we explore their application to trial matching. First, we design an LLM-based system which, given a patient's medical history as unstructured clinical text, evaluates whether that patient meets a set of inclusion criteria (also specified as free text). Our zero-shot system achieves state-of-the-art scores on the n2c2 2018 cohort selection benchmark. Second, we improve the data and cost efficiency of our method by identifying a prompting strategy which matches patients an order of magnitude faster and more cheaply than the status quo, and develop a two-stage retrieval pipeline that reduces the number of tokens processed by up to a third while retaining high performance. Third, we evaluate the interpretability of our system by having clinicians evaluate the natural language justifications generated by the LLM for each eligibility decision, and show that it can output coherent explanations for 97% of its correct decisions and 75% of its incorrect ones. Our results establish the feasibility of using LLMs to accelerate clinical trial operations.


title: Graph Neural Network and NER-Based Text Summarization

abstract: With the abundance of data and information in todays time, it is nearly impossible for man, or, even machine, to go through all of the data line by line. What one usually does is to try to skim through the lines and retain the absolutely important information, that in a more formal term is called summarization. Text summarization is an important task that aims to compress lengthy documents or articles into shorter, coherent representations while preserving the core information and meaning. This project introduces an innovative approach to text summarization, leveraging the capabilities of Graph Neural Networks (GNNs) and Named Entity Recognition (NER) systems. GNNs, with their exceptional ability to capture and process the relational data inherent in textual information, are adept at understanding the complex structures within large documents. Meanwhile, NER systems contribute by identifying and emphasizing key entities, ensuring that the summarization process maintains a focus on the most critical aspects of the text. By integrating these two technologies, our method aims to enhances the efficiency of summarization and also tries to ensures a high degree relevance in the condensed content. This project, therefore, offers a promising direction for handling the ever increasing volume of textual data in an information-saturated world.


title: Illuminate: A novel approach for depression detection with explainable analysis and proactive therapy using prompt engineering

abstract: This paper introduces a novel paradigm for depression detection and treatment using advanced Large Language Models (LLMs): Generative Pre-trained Transformer 4 (GPT-4), Llama 2 chat, and Gemini. These LLMs are fine-tuned with specialized prompts to diagnose, explain, and suggest therapeutic interventions for depression. A unique few-shot prompting method enhances the models' ability to analyze and explain depressive symptoms based on the DSM-5 criteria. In the interaction phase, the models engage in empathetic dialogue management, drawing from resources like PsychDB and a Cognitive Behavioral Therapy (CBT) Guide, fostering supportive interactions with individuals experiencing major depressive disorders. Additionally, the research introduces the Illuminate Database, enriched with various CBT modules, aiding in personalized therapy recommendations. The study evaluates LLM performance using metrics such as F1 scores, Precision, Recall, Cosine similarity, and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) across different test sets, demonstrating their effectiveness. This comprehensive approach blends cutting-edge AI with established psychological methods, offering new possibilities in mental health care and showcasing the potential of LLMs in revolutionizing depression diagnosis and treatment strategies.


title: Enhancing Textbook Question Answering Task with Large Language Models and Retrieval Augmented Generation

abstract: Textbook question answering (TQA) is a challenging task in artificial intelligence due to the complex nature of context and multimodal data. Although previous research has significantly improved the task, there are still some limitations including the models' weak reasoning and inability to capture contextual information in the lengthy context. The introduction of large language models (LLMs) has revolutionized the field of AI, however, directly applying LLMs often leads to inaccurate answers. This paper proposes a methodology that handle the out-of-domain scenario in TQA where concepts are spread across different lessons by incorporating the retrieval augmented generation (RAG) technique and utilize transfer learning to handle the long context and enhance reasoning abilities. Through supervised fine-tuning of the LLM model Llama-2 and the incorporation of RAG, our architecture outperforms the baseline, achieving a 4.12% accuracy improvement on validation set and 9.84% on test set for non-diagram multiple-choice questions.


title: Best Practices for Text Annotation with Large Language Models

abstract: Large Language Models (LLMs) have ushered in a new era of text annotation, as their ease-of-use, high accuracy, and relatively low costs have meant that their use has exploded in recent months. However, the rapid growth of the field has meant that LLM-based annotation has become something of an academic Wild West: the lack of established practices and standards has led to concerns about the quality and validity of research. Researchers have warned that the ostensible simplicity of LLMs can be misleading, as they are prone to bias, misunderstandings, and unreliable results. Recognizing the transformative potential of LLMs, this paper proposes a comprehensive set of standards and best practices for their reliable, reproducible, and ethical use. These guidelines span critical areas such as model selection, prompt engineering, structured prompting, prompt stability analysis, rigorous model validation, and the consideration of ethical and legal implications. The paper emphasizes the need for a structured, directed, and formalized approach to using LLMs, aiming to ensure the integrity and robustness of text annotation practices, and advocates for a nuanced and critical engagement with LLMs in social scientific research.


title: LB-KBQA: Large-language-model and BERT based Knowledge-Based Question and Answering System

abstract: Generative Artificial Intelligence (AI), because of its emergent abilities, has empowered various fields, one typical of which is large language models (LLMs). One of the typical application fields of Generative AI is large language models (LLMs), and the natural language understanding capability of LLM is dramatically improved when compared with conventional AI-based methods. The natural language understanding capability has always been a barrier to the intent recognition performance of the Knowledge-Based-Question-and-Answer (KBQA) system, which arises from linguistic diversity and the newly appeared intent. Conventional AI-based methods for intent recognition can be divided into semantic parsing-based and model-based approaches. However, both of the methods suffer from limited resources in intent recognition. To address this issue, we propose a novel KBQA system based on a Large Language Model(LLM) and BERT (LB-KBQA). With the help of generative AI, our proposed method could detect newly appeared intent and acquire new knowledge. In experiments on financial domain question answering, our model has demonstrated superior effectiveness.


title: Financial Report Chunking for Effective Retrieval Augmented Generation

abstract: Chunking information is a key step in Retrieval Augmented Generation (RAG). Current research primarily centers on paragraph-level chunking. This approach treats all texts as equal and neglects the information contained in the structure of documents. We propose an expanded approach to chunk documents by moving beyond mere paragraph-level chunking to chunk primary by structural element components of documents. Dissecting documents into these constituent elements creates a new way to chunk documents that yields the best chunk size without tuning. We introduce a novel framework that evaluates how chunking based on element types annotated by document understanding models contributes to the overall context and accuracy of the information retrieved. We also demonstrate how this approach impacts RAG assisted Question & Answer task performance. Our research includes a comprehensive analysis of various element types, their role in effective information retrieval, and the impact they have on the quality of RAG outputs. Findings support that element type based chunking largely improve RAG results on financial reporting. Through this research, we are also able to answer how to uncover highly accurate RAG.


title: TexShape: Information Theoretic Sentence Embedding for Language Models

abstract: With the exponential growth in data volume and the emergence of data-intensive applications, particularly in the field of machine learning, concerns related to resource utilization, privacy, and fairness have become paramount. This paper focuses on the textual domain of data and addresses challenges regarding encoding sentences to their optimized representations through the lens of information-theory. In particular, we use empirical estimates of mutual information, using the Donsker-Varadhan definition of Kullback-Leibler divergence. Our approach leverages this estimation to train an information-theoretic sentence embedding, called TexShape, for (task-based) data compression or for filtering out sensitive information, enhancing privacy and fairness. In this study, we employ a benchmark language model for initial text representation, complemented by neural networks for information-theoretic compression and mutual information estimations. Our experiments demonstrate significant advancements in preserving maximal targeted information and minimal sensitive information over adverse compression ratios, in terms of predictive accuracy of downstream models that are trained using the compressed data.


title: Personalized Language Modeling from Personalized Human Feedback

abstract: Reinforcement Learning from Human Feedback (RLHF) is the current dominating framework to fine-tune large language models to better align with human preferences. However, the underlying premise of algorithms developed under this framework can be problematic when user preferences encoded in human feedback are diverse. In this work, we aim to address this problem by developing methods for building personalized language models. We first formally introduce the task of learning from personalized human feedback and explain why vanilla RLHF can be problematic in this context. We then propose a general Personalized-RLHF (P-RLHF) framework, which requires one to jointly learn a user model and a language (or reward) model. The user model takes in user information and outputs user representations. Its structure encodes our assumptions about user preferences underlying the feedback data. We develop new learning objectives for personalized reward modeling and personalized Direct Preference Optimization. To demonstrate the efficacy of our method, we test it on real-world text summarization data with annotated preferences and annotator information. We fine-tune GPT-J 6B to obtain personalized language (and reward) models, which outperform non-personalized models in terms of aligning with individual preferences.


title: The post-Newtonian metric of a self-gravitating and collapsing thin spherical shell

abstract: We calculate the metric for the thought experiment of a self-gravitating and collapsing infinitely-thin spherical shell, in the first post-Newtonian approximation for general relativity. The post-Newtonian dynamic equations of the test particles inside and outside the shell are achieved respectively.


title: CADReN: Contextual Anchor-Driven Relational Network for Controllable Cross-Graphs Node Importance Estimation

abstract: Node Importance Estimation (NIE) is crucial for integrating external information into Large Language Models through Retriever-Augmented Generation. Traditional methods, focusing on static, single-graph characteristics, lack adaptability to new graphs and user-specific requirements. CADReN, our proposed method, addresses these limitations by introducing a Contextual Anchor (CA) mechanism. This approach enables the network to assess node importance relative to the CA, considering both structural and semantic features within Knowledge Graphs (KGs). Extensive experiments show that CADReN achieves better performance in cross-graph NIE task, with zero-shot prediction ability. CADReN is also proven to match the performance of previous models on single-graph NIE task. Additionally, we introduce and opensource two new datasets, RIC200 and WK1K, specifically designed for cross-graph NIE research, providing a valuable resource for future developments in this domain.


title: LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256K

abstract: State-of-the-art large language models (LLMs) are now claiming remarkable supported context lengths of 256k or even more. In contrast, the average context lengths of mainstream benchmarks are insufficient (5k-21k), and they suffer from potential knowledge leakage and inaccurate metrics, resulting in biased evaluation. This paper introduces LV-Eval, a challenging long-context benchmark with five length levels (16k, 32k, 64k, 128k, and 256k) reaching up to 256k words. LV-Eval features two main tasks, single-hop QA and multi-hop QA, comprising 11 bilingual datasets. The design of LV-Eval has incorporated three key techniques, namely confusing facts insertion, keyword and phrase replacement, and keyword-recall-based metric design. The advantages of LV-Eval include controllable evaluation across different context lengths, challenging test instances with confusing facts, mitigated knowledge leakage, and more objective evaluations. We evaluate 10 LLMs on LV-Eval and conduct ablation studies on the techniques used in LV-Eval construction. The results reveal that: (i) Commercial LLMs generally outperform open-source LLMs when evaluated within length levels shorter than their claimed context length. However, their overall performance is surpassed by open-source LLMs with longer context lengths. (ii) Extremely long-context LLMs, such as Yi-6B-200k, exhibit a relatively gentle degradation of performance, but their absolute performances may not necessarily be higher than those of LLMs with shorter context lengths. (iii) LLMs' performances can significantly degrade in the presence of confusing information, especially in the pressure test of "needle in a haystack". (iv) Issues related to knowledge leakage and inaccurate metrics introduce bias in evaluation, and these concerns are alleviated in LV-Eval. All datasets and evaluation codes are released at: https://github.com/infinigence/LVEval.


title: LtU-ILI: An All-in-One Framework for Implicit Inference in Astrophysics and Cosmology

abstract: This paper presents the Learning the Universe Implicit Likelihood Inference (LtU-ILI) pipeline, a codebase for rapid, user-friendly, and cutting-edge machine learning (ML) inference in astrophysics and cosmology. The pipeline includes software for implementing various neural architectures, training schema, priors, and density estimators in a manner easily adaptable to any research workflow. It includes comprehensive validation metrics to assess posterior estimate coverage, enhancing the reliability of inferred results. Additionally, the pipeline is easily parallelizable, designed for efficient exploration of modeling hyperparameters. To demonstrate its capabilities, we present real applications across a range of astrophysics and cosmology problems, such as: estimating galaxy cluster masses from X-ray photometry; inferring cosmology from matter power spectra and halo point clouds; characterising progenitors in gravitational wave signals; capturing physical dust parameters from galaxy colors and luminosities; and establishing properties of semi-analytic models of galaxy formation. We also include exhaustive benchmarking and comparisons of all implemented methods as well as discussions about the challenges and pitfalls of ML inference in astronomical sciences. All code and examples are made publicly available at https://github.com/maho3/ltu-ili.

categories: astro-ph.IM astro-ph.CO astro-ph.GA cs.LG


title: SceMQA: A Scientific College Entrance Level Multimodal Question Answering Benchmark

abstract: The paper introduces SceMQA, a novel benchmark for scientific multimodal question answering at the college entrance level. It addresses a critical educational phase often overlooked in existing benchmarks, spanning high school to pre-college levels. SceMQA focuses on core science subjects including Mathematics, Physics, Chemistry, and Biology. It features a blend of multiple-choice and free-response formats, ensuring a comprehensive evaluation of AI models' abilities. Additionally, our benchmark provides specific knowledge points for each problem and detailed explanations for each answer. SceMQA also uniquely presents problems with identical contexts but varied questions to facilitate a more thorough and accurate assessment of reasoning capabilities. In the experiment, we evaluate both open-source and close-source state-of-the-art Multimodal Large Language Models (MLLMs), across various experimental settings. The results show that further research and development are needed in developing more capable MLLM, as highlighted by only 50% to 60% accuracy achieved by the strongest models. Our benchmark and analysis will be available at https://scemqa.github.io/


title: Crustal velocity and strain rate fields in the Balearic Islands based on continuous GPS time series from the XGAIB network (2010-2013)

abstract: In this paper, we present a first estimation, using the GIPSY-OASIS software, of the crustal velocity and strain rate fields in the Balearic Islands (Spain), based on continuous GPS observations from the XGAIB network spanning the period 2010-2013. The XGAIB network consists of nine permanent, widely distributed stations that have operated continuously since 2010. In this paper, we describe the XGAIB network and the CGPS data processing and present our principle results in terms of the position time series and velocities of all of the sites, which were observed for more than three and a half years. In addition, strain tensors were estimated from the velocity field to obtain the first realistic crustal deformation model of the archipelago. The strains exhibit gradual variation across the Balearic Islands, from WNW-ESE extension in the southwest (Ibiza and Formentera) to NW-SE compression in the northeast (Menorca). These results constitute an advance in our knowledge of the tectonics of the western Mediterranean region.


title: Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domains

abstract: Large Language Models (LLMs) have demonstrated remarkable proficiency in understanding and generating natural language. However, their capabilities wane in highly specialized domains underrepresented in the pretraining corpus, such as physical and biomedical sciences. This work explores how to repurpose general LLMs into effective task solvers for specialized domains. We introduce a novel, model-agnostic framework for learning custom input tags, which are parameterized as continuous vectors appended to the LLM's embedding layer, to condition the LLM. We design two types of input tags: domain tags are used to delimit specialized representations (e.g., chemical formulas) and provide domain-relevant context; function tags are used to represent specific functions (e.g., predicting molecular properties) and compress function-solving instructions. We develop a three-stage protocol to learn these tags using auxiliary data and domain knowledge. By explicitly disentangling task domains from task functions, our method enables zero-shot generalization to unseen problems through diverse combinations of the input tags. It also boosts LLM's performance in various specialized domains, such as predicting protein or chemical properties and modeling drug-target interactions, outperforming expert models tailored to these tasks.


title: Tensor Completion via Integer Optimization

abstract: The main challenge with the tensor completion problem is a fundamental tension between computation power and the information-theoretic sample complexity rate. Past approaches either achieve the information-theoretic rate but lack practical algorithms to compute the corresponding solution, or have polynomial-time algorithms that require an exponentially-larger number of samples for low estimation error. This paper develops a novel tensor completion algorithm that resolves this tension by achieving both provable convergence (in numerical tolerance) in a linear number of oracle steps and the information-theoretic rate. Our approach formulates tensor completion as a convex optimization problem constrained using a gauge-based tensor norm, which is defined in a way that allows the use of integer linear optimization to solve linear separation problems over the unit-ball in this new norm. Adaptations based on this insight are incorporated into a Frank-Wolfe variant to build our algorithm. We show our algorithm scales-well using numerical experiments on tensors with up to ten million entries.


title: The Foundations of Computational Management: A Systematic Approach to Task Automation for the Integration of Artificial Intelligence into Existing Workflows

abstract: Driven by the rapid ascent of artificial intelligence (AI), organizations are at the epicenter of a seismic shift, facing a crucial question: How can AI be successfully integrated into existing operations? To help answer it, manage expectations and mitigate frustration, this article introduces Computational Management, a systematic approach to task automation for enhancing the ability of organizations to harness AI's potential within existing workflows. Computational Management acts as a bridge between the strategic insights of management science with the analytical rigor of computational thinking. The article offers three easy step-by-step procedures to begin the process of implementing AI within a workflow. Such procedures focus on task (re)formulation, on the assessment of the automation potential of tasks, on the completion of task specification templates for AI selection and adaptation. Included in the article there are manual and automated methods, with prompt suggestions for publicly available LLMs, to complete these three procedures. The first procedure, task (re)formulation, focuses on breaking down work activities into basic units, so they can be completed by one agent, involve a single well-defined action, and produce a distinct outcome. The second, allows the assessment of the granular task and its suitability for automation, using the Task Automation Index to rank tasks based on whether they have standardized input, well-defined rules, repetitiveness, data dependency, and objective outputs. The third, focuses on a task specification template which details information on 16 critical components of tasks, and can be used as a checklist to select or adapt the most suitable AI solution for integration into existing workflows. Computational Management provides a roadmap and a toolkit for humans and AI to thrive together, while enhancing organizational efficiency and innovation.


title: Liouvillian Solutions of Third Order Differential Equations

abstract: Consider a third order linear differential equation $L(f)=0$, where $L\in\mathbb{Q}(z)[\partial_z]$. We design an algorithm computing the Liouvillian solutions of $L(f)=0$. The reducible cases devolve to the classical case of second order operators, and in the irreducible cases, only finitely many differential Galois groups are possible. The differential Galois group is obtained through optimized computations of invariants and semi-invariants, and if solvable, the solutions are returned as pullbacks and gauge transformations of algebraic generalized hypergeometric function ${}_3F_2$. The computation time is practical for reasonable size operators.


title: A Bandit Approach with Evolutionary Operators for Model Selection

abstract: This work formulates model selection as an infinite-armed bandit problem, namely, a problem in which a decision maker iteratively selects one of an infinite number of fixed choices (i.e., arms) when the properties of each choice are only partially known at the time of allocation and may become better understood over time, via the attainment of rewards.Here, the arms are machine learning models to train and selecting an arm corresponds to a partial training of the model (resource allocation).The reward is the accuracy of the selected model after its partial training.We aim to identify the best model at the end of a finite number of resource allocations and thus consider the best arm identification setup. We propose the algorithm Mutant-UCB that incorporates operators from evolutionary algorithms into the UCB-E (Upper Confidence Bound Exploration) bandit algorithm introduced by Audiber et al.Tests carried out on three open source image classification data sets attest to the relevance of this novel combining approach, which outperforms the state-of-the-art for a fixed budget.


title: Online Learning Approach for Survival Analysis

abstract: We introduce an online mathematical framework for survival analysis, allowing real time adaptation to dynamic environments and censored data. This framework enables the estimation of event time distributions through an optimal second order online convex optimization algorithm-Online Newton Step (ONS). This approach, previously unexplored, presents substantial advantages, including explicit algorithms with non-asymptotic convergence guarantees. Moreover, we analyze the selection of ONS hyperparameters, which depends on the exp-concavity property and has a significant influence on the regret bound. We propose a stochastic approach that guarantees logarithmic stochastic regret for ONS. Additionally, we introduce an adaptive aggregation method that ensures robustness in hyperparameter selection while maintaining fast regret bounds. The findings of this paper can extend beyond the survival analysis field, and are relevant for any case characterized by poor exp-concavity and unstable ONS. Finally, these assertions are illustrated by simulation experiments.


title: Compressing Deep Reinforcement Learning Networks with a Dynamic Structured Pruning Method for Autonomous Driving

abstract: Deep reinforcement learning (DRL) has shown remarkable success in complex autonomous driving scenarios. However, DRL models inevitably bring high memory consumption and computation, which hinders their wide deployment in resource-limited autonomous driving devices. Structured Pruning has been recognized as a useful method to compress and accelerate DRL models, but it is still challenging to estimate the contribution of a parameter (i.e., neuron) to DRL models. In this paper, we introduce a novel dynamic structured pruning approach that gradually removes a DRL model's unimportant neurons during the training stage. Our method consists of two steps, i.e. training DRL models with a group sparse regularizer and removing unimportant neurons with a dynamic pruning threshold. To efficiently train the DRL model with a small number of important neurons, we employ a neuron-importance group sparse regularizer. In contrast to conventional regularizers, this regularizer imposes a penalty on redundant groups of neurons that do not significantly influence the output of the DRL model. Furthermore, we design a novel structured pruning strategy to dynamically determine the pruning threshold and gradually remove unimportant neurons with a binary mask. Therefore, our method can remove not only redundant groups of neurons of the DRL model but also achieve high and robust performance. Experimental results show that the proposed method is competitive with existing DRL pruning methods on discrete control environments (i.e., CartPole-v1 and LunarLander-v2) and MuJoCo continuous environments (i.e., Hopper-v3 and Walker2D-v3). Specifically, our method effectively compresses $93\%$ neurons and $96\%$ weights of the DRL model in four challenging DRL environments with slight accuracy degradation.


title: ApiQ: Finetuning of 2-Bit Quantized Large Language Model

abstract: Memory-efficient finetuning of large language models (LLMs) has recently attracted huge attention with the increasing size of LLMs, primarily due to the constraints posed by GPU memory limitations and the effectiveness of these methods compared to full finetuning. Despite the advancements, current strategies for memory-efficient finetuning, such as QLoRA, exhibit inconsistent performance across diverse bit-width quantizations and multifaceted tasks. This inconsistency largely stems from the detrimental impact of the quantization process on preserved knowledge, leading to catastrophic forgetting and undermining the utilization of pretrained models for finetuning purposes. In this work, we introduce a novel quantization framework, ApiQ, designed to restore the lost information from quantization by concurrently initializing the LoRA components and quantizing the weights of LLMs. This approach ensures the maintenance of the original LLM's activation precision while mitigating the error propagation from shallower into deeper layers. Through comprehensive evaluations conducted on a spectrum of language tasks with various LLMs, ApiQ demonstrably minimizes activation error during quantization. Consequently, it consistently achieves superior finetuning results across various bit-widths.


title: Cost Optimized Scheduling in Modular Electrolysis Plants

abstract: In response to the global shift towards renewable energy resources, the production of green hydrogen through electrolysis is emerging as a promising solution. Modular electrolysis plants, designed for flexibility and scalability, offer a dynamic response to the increasing demand for hydrogen while accommodating the fluctuations inherent in renewable energy sources. However, optimizing their operation is challenging, especially when a large number of electrolysis modules needs to be coordinated, each with potentially different characteristics. To address these challenges, this paper presents a decentralized scheduling model to optimize the operation of modular electrolysis plants using the Alternating Direction Method of Multipliers. The model aims to balance hydrogen production with fluctuating demand, to minimize the marginal Levelized Cost of Hydrogen (mLCOH), and to ensure adaptability to operational disturbances. A case study validates the accuracy of the model in calculating mLCOH values under nominal load conditions and demonstrates its responsiveness to dynamic changes, such as electrolyzer module malfunctions and scale-up scenarios.


title: FlowPG: Action-constrained Policy Gradient with Normalizing Flows

abstract: Action-constrained reinforcement learning (ACRL) is a popular approach for solving safety-critical and resource-allocation related decision making problems. A major challenge in ACRL is to ensure agent taking a valid action satisfying constraints in each RL step. Commonly used approach of using a projection layer on top of the policy network requires solving an optimization program which can result in longer training time, slow convergence, and zero gradient problem. To address this, first we use a normalizing flow model to learn an invertible, differentiable mapping between the feasible action space and the support of a simple distribution on a latent variable, such as Gaussian. Second, learning the flow model requires sampling from the feasible action space, which is also challenging. We develop multiple methods, based on Hamiltonian Monte-Carlo and probabilistic sentential decision diagrams for such action sampling for convex and non-convex constraints. Third, we integrate the learned normalizing flow with the DDPG algorithm. By design, a well-trained normalizing flow will transform policy output into a valid action without requiring an optimization solver. Empirically, our approach results in significantly fewer constraint violations (upto an order-of-magnitude for several instances) and is multiple times faster on a variety of continuous control tasks.


title: Designing deep neural networks for driver intention recognition

abstract: Driver intention recognition studies increasingly rely on deep neural networks. Deep neural networks have achieved top performance for many different tasks, but it is not a common practice to explicitly analyse the complexity and performance of the network's architecture. Therefore, this paper applies neural architecture search to investigate the effects of the deep neural network architecture on a real-world safety critical application with limited computational capabilities. We explore a pre-defined search space for three deep neural network layer types that are capable to handle sequential data (a long-short term memory, temporal convolution, and a time-series transformer layer), and the influence of different data fusion strategies on the driver intention recognition performance. A set of eight search strategies are evaluated for two driver intention recognition datasets. For the two datasets, we observed that there is no search strategy clearly sampling better deep neural network architectures. However, performing an architecture search does improve the model performance compared to the original manually designed networks. Furthermore, we observe no relation between increased model complexity and higher driver intention recognition performance. The result indicate that multiple architectures yield similar performance, regardless of the deep neural network layer type or fusion strategy.


title: CrashFormer: A Multimodal Architecture to Predict the Risk of Crash

abstract: Reducing traffic accidents is a crucial global public safety concern. Accident prediction is key to improving traffic safety, enabling proactive measures to be taken before a crash occurs, and informing safety policies, regulations, and targeted interventions. Despite numerous studies on accident prediction over the past decades, many have limitations in terms of generalizability, reproducibility, or feasibility for practical use due to input data or problem formulation. To address existing shortcomings, we propose CrashFormer, a multi-modal architecture that utilizes comprehensive (but relatively easy to obtain) inputs such as the history of accidents, weather information, map images, and demographic information. The model predicts the future risk of accidents on a reasonably acceptable cadence (i.e., every six hours) for a geographical location of 5.161 square kilometers. CrashFormer is composed of five components: a sequential encoder to utilize historical accidents and weather data, an image encoder to use map imagery data, a raw data encoder to utilize demographic information, a feature fusion module for aggregating the encoded features, and a classifier that accepts the aggregated data and makes predictions accordingly. Results from extensive real-world experiments in 10 major US cities show that CrashFormer outperforms state-of-the-art sequential and non-sequential models by 1.8% in F1-score on average when using ``sparse'' input data.


title: Is the price right? Reconceptualizing price and income elasticity to anticipate price perception issues

abstract: Price perception by consumers represents a challenge to the ability of a business to correctly and profitably price and sell their products or services in a given market and any new target market. Complicating the perception of prices is the dynamics of price and income elasticity, both of which are key for estimating demand. This article proposes a novel and elegant identity that conceptualizes elasticity as a means of quantifying the potential for price perception problems in a market. Elasticity studies from 1990 to 2023 (n=30) were sampled to evaluate the relationship between price and income elasticity for various consumer commodities using the identity. The results suggest that, given known price and income elasticity values, a business can anticipate pricing perception problems in advance and address the potential for damaging distortion of their value proposition. Further, the business can use this insight to correctly choose a strategic pricing approach.


title: Estimating On-road Transportation Carbon Emissions from Open Data of Road Network and Origin-destination Flow Data

abstract: Accounting for over 20% of the total carbon emissions, the precise estimation of on-road transportation carbon emissions is crucial for carbon emission monitoring and efficient mitigation policy formulation. However, existing estimation methods typically depend on hard-to-collect individual statistics of vehicle miles traveled to calculate emissions, thereby suffering from high data collection difficulty. To relieve this issue by utilizing the strong pattern recognition of artificial intelligence, we incorporate two sources of open data representative of the transportation demand and capacity factors, the origin-destination (OD) flow data and the road network data, to build a hierarchical heterogeneous graph learning method for on-road carbon emission estimation (HENCE). Specifically, a hierarchical graph consisting of the road network level, community level, and region level is constructed to model the multi-scale road network-based connectivity and travel connection between spatial areas. Heterogeneous graphs consisting of OD links and spatial links are further built at both the community level and region level to capture the intrinsic interactions between travel demand and road network accessibility. Extensive experiments on two large-scale real-world datasets demonstrate HENCE's effectiveness and superiority with R-squared exceeding 0.75 and outperforming baselines by 9.60% on average, validating its success in pioneering the use of artificial intelligence to empower carbon emission management and sustainability development. The implementation codes are available at this link: https://github.com/tsinghua-fib-lab/HENCE.


abstract: Trust plays an essential role in an individual's decision-making. Traditional trust prediction models rely on pairwise correlations to infer potential relationships between users. However, in the real world, interactions between users are usually complicated rather than pairwise only. Hypergraphs offer a flexible approach to modeling these complex high-order correlations (not just pairwise connections), since hypergraphs can leverage hyperedeges to link more than two nodes. However, most hypergraph-based methods are generic and cannot be well applied to the trust prediction task. In this paper, we propose an Adaptive Hypergraph Network for Trust Prediction (AHNTP), a novel approach that improves trust prediction accuracy by using higher-order correlations. AHNTP utilizes Motif-based PageRank to capture high-order social influence information. In addition, it constructs hypergroups from both node-level and structure-level attributes to incorporate complex correlation information. Furthermore, AHNTP leverages adaptive hypergraph Graph Convolutional Network (GCN) layers and multilayer perceptrons (MLPs) to generate comprehensive user embeddings, facilitating trust relationship prediction. To enhance model generalization and robustness, we introduce a novel supervised contrastive learning loss for optimization. Extensive experiments demonstrate the superiority of our model over the state-of-the-art approaches in terms of trust prediction accuracy. The source code of this work can be accessed via https://github.com/Sherry-XU1995/AHNTP.


title: Non-convergence to global minimizers for Adam and stochastic gradient descent optimization and constructions of local minimizers in the training of artificial neural networks

abstract: Stochastic gradient descent (SGD) optimization methods such as the plain vanilla SGD method and the popular Adam optimizer are nowadays the method of choice in the training of artificial neural networks (ANNs). Despite the remarkable success of SGD methods in the ANN training in numerical simulations, it remains in essentially all practical relevant scenarios an open problem to rigorously explain why SGD methods seem to succeed to train ANNs. In particular, in most practically relevant supervised learning problems, it seems that SGD methods do with high probability not converge to global minimizers in the optimization landscape of the ANN training problem. Nevertheless, it remains an open problem of research to disprove the convergence of SGD methods to global minimizers. In this work we solve this research problem in the situation of shallow ANNs with the rectified linear unit (ReLU) and related activations with the standard mean square error loss by disproving in the training of such ANNs that SGD methods (such as the plain vanilla SGD, the momentum SGD, the AdaGrad, the RMSprop, and the Adam optimizers) can find a global minimizer with high probability. Even stronger, we reveal in the training of such ANNs that SGD methods do with high probability fail to converge to global minimizers in the optimization landscape. The findings of this work do, however, not disprove that SGD methods succeed to train ANNs since they do not exclude the possibility that SGD methods find good local minimizers whose risk values are close to the risk values of the global minimizers. In this context, another key contribution of this work is to establish the existence of a hierarchical structure of local minimizers with distinct risk values in the optimization landscape of ANN training problems with ReLU and related activations.


title: What About the Data? A Mapping Study on Data Engineering for AI Systems

abstract: AI systems cannot exist without data. Now that AI models (data science and AI) have matured and are readily available to apply in practice, most organizations struggle with the data infrastructure to do so. There is a growing need for data engineers that know how to prepare data for AI systems or that can setup enterprise-wide data architectures for analytical projects. But until now, the data engineering part of AI engineering has not been getting much attention, in favor of discussing the modeling part. In this paper we aim to change this by perform a mapping study on data engineering for AI systems, i.e., AI data engineering. We found 25 relevant papers between January 2019 and June 2023, explaining AI data engineering activities. We identify which life cycle phases are covered, which technical solutions or architectures are proposed and which lessons learned are presented. We end by an overall discussion of the papers with implications for practitioners and researchers. This paper creates an overview of the body of knowledge on data engineering for AI. This overview is useful for practitioners to identify solutions and best practices as well as for researchers to identify gaps.


title: Stellar halo density with LAMOST K and M giants

abstract: AIMS. We derive the morphology of the stellar component in the outer halo volume, and search for possible overdensities due to substructures therein. METHODS. We made use of some of the data releases of the spectroscopic survey LAMOST DR8-DR9 in tandem with distance determinations for two subsamples, that is, of K-giants and M-giants, respectively, making up 60,000 stars. These distance are obtained through Bayesian techniques that derive absolute magnitudes as a function of measured spectroscopic parameters. Our calculation of the density from these catalogues requires: (1) derivation of the selection function; and (2) a correction for the convolution of the distance errors, which we carried out with Lucy's inversion of the corresponding integral equation. RESULTS. The stellar density distribution of the outer halo (distance to the Galactic centre, $r_G$, of between 25 and 90 kpc) is a smooth monotonously decreasing function with a dependence of approximately $\rho \propto r_G^{-n}$, with $n=4.6\pm 0.4$ for K-giants and $n=4.5\pm 0.2$ for M-giants, and with a insignificant oblateness. The value of $n$ is independent of the angular distance to the Sagittarius tidal stream plane, which is what would be expected if such a stream did not exist in the anticenter positions or had a negligible imprint in the density distribution in the outer halo. Apart from random fluctuations or minor anomalies in some lines of sight, we do not see substructures superimposed in the outer halo volume within the resolution that we are using and limited by the error bars. This constrains the mass of over- and under-densities in the outer halo to be of $\lesssim 10^3$ M$_\odot $/deg$^2$, whereas the total mass of the stellar halo, including inner and outer parts, is $\sim 7\times 10^8$ M$_\odot $.


title: Enhancement of Bengali OCR by Specialized Models and Advanced Techniques for Diverse Document Types

abstract: This research paper presents a unique Bengali OCR system with some capabilities. The system excels in reconstructing document layouts while preserving structure, alignment, and images. It incorporates advanced image and signature detection for accurate extraction. Specialized models for word segmentation cater to diverse document types, including computer-composed, letterpress, typewriter, and handwritten documents. The system handles static and dynamic handwritten inputs, recognizing various writing styles. Furthermore, it has the ability to recognize compound characters in Bengali. Extensive data collection efforts provide a diverse corpus, while advanced technical components optimize character and word recognition. Additional contributions include image, logo, signature and table recognition, perspective correction, layout reconstruction, and a queuing module for efficient and scalable processing. The system demonstrates outstanding performance in efficient and accurate text extraction and analysis.


title: Threats and Limitations of Terrestrial Broadcast Attacks

abstract: The DVB standard does not mandate the use of authentication and integrity protection for transport streams. This allows malicious third parties to replace legitimate broadcasts by overpowering terrestrial transmissions. The rogue signal can then deliver a malicious broadcast stream to exploit security vulnerabilities on Smart TVs (STVs) in range. We implemented a proof-of-concept attack based on a malicious Hybrid Broadcast Broadband TV app, able to acquire permanent system-level access to an STV over the air, in less than 10 s. These attacks, however, are severely limited in range due to required co-channel protection ratios (CCPRs), which is in direct contradiction to previous publications. We present evidence for these limitations in form of laboratory experiments, extensive simulations, and field measurements. To this end, we developed an automated, low-cost method for CCPR determination, as well as a method for non-disruptive attack range measurements based on a gap filler and the resulting channel impulse response.


title: What's documented in AI? Systematic Analysis of 32K AI Model Cards

abstract: The rapid proliferation of AI models has underscored the importance of thorough documentation, as it enables users to understand, trust, and effectively utilize these models in various applications. Although developers are encouraged to produce model cards, it's not clear how much information or what information these cards contain. In this study, we conduct a comprehensive analysis of 32,111 AI model documentations on Hugging Face, a leading platform for distributing and deploying AI models. Our investigation sheds light on the prevailing model card documentation practices. Most of the AI models with substantial downloads provide model cards, though the cards have uneven informativeness. We find that sections addressing environmental impact, limitations, and evaluation exhibit the lowest filled-out rates, while the training section is the most consistently filled-out. We analyze the content of each section to characterize practitioners' priorities. Interestingly, there are substantial discussions of data, sometimes with equal or even greater emphasis than the model itself. To evaluate the impact of model cards, we conducted an intervention study by adding detailed model cards to 42 popular models which had no or sparse model cards previously. We find that adding model cards is moderately correlated with an increase weekly download rates. Our study opens up a new perspective for analyzing community norms and practices for model documentation through large-scale data science and linguistics analysis.


title: Approximate Keys and Functional Dependencies in Incomplete Databases With Limited Domains-Algorithmic Perspective

abstract: A possible world of an incomplete database table is obtained by imputing values from the attributes (infinite) domain to the place of \texttt{NULL} s. A table satisfies a possible key or possible functional dependency constraint if there exists a possible world of the table that satisfies the given key or functional dependency constraint. A certain key or functional dependency is satisfied by a table if all of its possible worlds satisfy the constraint. Recently, an intermediate concept was introduced. A strongly possible key or functional dependency is satisfied by a table if there exists a strongly possible world that satisfies the key or functional dependency. A strongly possible world is obtained by imputing values from the active domain of the attributes, that is from the values appearing in the table. In the present paper, we study approximation measures of strongly possible keys and FDs. Measure $g_3$ is the ratio of the minimum number of tuples to be removed in order that the remaining table satisfies the constraint. We introduce a new measure $g_5$, the ratio of the minimum number of tuples to be added to the table so the result satisfies the constraint. $g_5$ is meaningful because the addition of tuples may extend the active domains. We prove that if $g_5$ can be defined for a table and a constraint, then the $g_3$ value is always an upper bound of the $g_5$ value. However, the two measures are independent of each other in the sense that for any rational number $0\le\frac{p}{q}<1$ there are tables of an arbitrarily large number of rows and a constant number of columns that satisfy $g_3-g_5=\frac{p}{q}$. A possible world is obtained usually by adding many new values not occurring in the table before. The measure $g_5$ measures the smallest possible distortion of the active domains. We study complexity of determining these approximate measures.


title: Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications

abstract: Large language models (LLMs) show inherent brittleness in their safety mechanisms, as evidenced by their susceptibility to jailbreaking and even non-malicious fine-tuning. This study explores this brittleness of safety alignment by leveraging pruning and low-rank modifications. We develop methods to identify critical regions that are vital for safety guardrails, and that are disentangled from utility-relevant regions at both the neuron and rank levels. Surprisingly, the isolated regions we find are sparse, comprising about $3\%$ at the parameter level and $2.5\%$ at the rank level. Removing these regions compromises safety without significantly impacting utility, corroborating the inherent brittleness of the model's safety mechanisms. Moreover, we show that LLMs remain vulnerable to low-cost fine-tuning attacks even when modifications to the safety-critical regions are restricted. These findings underscore the urgent need for more robust safety strategies in LLMs.


title: Ultra-Short Pulse Biphoton Source in Lithium Niobate Nanophotonics at 2$\text{\mu}$m

abstract: Photonics offers unique capabilities for quantum information processing (QIP) such as room-temperature operation, the scalability of nanophotonics, and access to ultrabroad bandwidths and consequently ultrafast operation. Ultrashort-pulse sources of quantum states in nanophotonics are an important building block for achieving scalable ultrafast QIP, however, their demonstrations so far have been sparse. Here, we demonstrate a femtosecond biphoton source in dispersion-engineered periodically poled lithium niobate nanophotonics. We measure 17 THz of bandwidth for the source centered at 2.09 \textmu m, corresponding to a few optical cycles, with a brightness of 8.8 GHz/mW. Our results open new paths towards realization of ultrafast nanophotonic QIP.


title: A Resource Model For Neural Scaling Law

abstract: Neural scaling laws characterize how model performance improves as the model size scales up. Inspired by empirical observations, we introduce a resource model of neural scaling. A task is usually composite hence can be decomposed into many subtasks, which compete for resources (measured by the number of neurons allocated to subtasks). On toy problems, we empirically find that: (1) The loss of a subtask is inversely proportional to its allocated neurons. (2) When multiple subtasks are present in a composite task, the resources acquired by each subtask uniformly grow as models get larger, keeping the ratios of acquired resources constants. We hypothesize these findings to be generally true and build a model to predict neural scaling laws for general composite tasks, which successfully replicates the neural scaling law of Chinchilla models reported in arXiv:2203.15556. We believe that the notion of resource used in this paper will be a useful tool for characterizing and diagnosing neural networks.


title: The cold molecular gas in z$\gtrsim$6 quasar host galaxies

abstract: Probing the molecular gas reservoirs of z>~6 quasar (QSO) host galaxies is fundamental to understanding the coevolution of star formation and black hole growth in these extreme systems. Yet, there is still an inhomogeneous coverage of molecular gas tracers. To measure the average excitation and mass of the molecular gas reservoirs in the brightest z>6.5 QSO hosts, we combined new observations of CO(2-1) emission with existing observations of CO(6-5), CO(7-6), [C I], [C II], and dust-continuum emission. We reduced and analysed the VLA observations of CO(2-1) in three z=6.5-6.9 QSO hosts -- the highest redshift observations of CO(2-1) to date. By combining these with the nine z=5.7-6.4 QSO hosts for which CO(2-1) has already been observed, we studied the spread in molecular gas masses and CO excitation. Two of our three QSOs, were undetected in CO(2-1), implying more highly excited CO than in the z=6.4 QSO J1148+5251. We detected CO(2-1) at $5.1\sigma$ for our highest-redshift target, J2348-3054, yielding a molecular gas mass of $(1.2\pm0.2)\times 10^{10}\, \mathrm{M}_\odot$. This molecular gas mass is equivalent to the lower limit on the dynamical mass measured from resolved [C II] observations, implying little mass in stars or neutral gas within the [C II]-emitting region. On average, these QSO hosts have far higher CO(6-5)-, CO(7-6)-, and [C II] vs CO(2-1) line ratios than local AGN hosts; with a mean CO(6-5)-to-CO(1-0) line luminosity ratio of $r_{6,1}=0.9\pm0.2$. Our new CO(2-1) observations show that even at 780 Myr after the Big Bang, QSO host galaxies can already have molecular gas masses of $10^{10}$ M$_\odot$, consistent with a picture in which these z>6 QSOs reside in massive starbursts that are coevolving with the supermassive black holes. Our results imply the presence of extremely dense and warm molecular gas reservoirs illuminated by strong interstellar radiation fields.


title: Hot spaces with positive cosmological constant in the canonical ensemble: de Sitter solution, Schwarzschild-de Sitter black hole, and Nariai universe

abstract: In a space with positive cosmological constant $\Lambda$, we consider a black hole surrounded by a heat reservoir at radius $R$ and temperature $T$, i.e., we analyze the Schwarzschild-de Sitter black hole in a cavity. We use the Euclidean path integral approach to quantum gravity to study its canonical ensemble and thermodynamics. We give the action, energy, entropy, temperature, and heat capacity. $T$, $\Lambda$, the black hole radius $r_+$, and the cosmological horizon radius $r_{\rm c}$, are gauged in $R$ units to $RT$, $\Lambda R^2$, $\frac{r_+}{R}$, and $\frac{r_{\rm c}}{R}$. The whole extension of $\Lambda R^2$, $0\leq\Lambda R^2\leq 3$, is divided into three ranges. The first, $0\leq\Lambda R^2<1$, includes York's Schwarzschild black holes. The second range, $\Lambda R^2=1$, opens up a folder of Nariai universes. The third range, $1<\Lambda R^2\leq 3$, is unusual. One feature here is that it interchanges the cosmological horizon with the black hole horizon. The end point, $\Lambda R^2=3$, only existing for infinite $RT$, is a cavity filled with de Sitter space, except for a singularity, with the cosmological horizon coinciding with the reservoir. For the three ranges, for low temperatures, there are no black holes and no Nariai universes, the space is hot de Sitter. The value of $RT$ that divides the nonexistence from existence of black holes or Nariai universes, depends on $\Lambda R^2$. For each $\Lambda R^2\neq1$, for high temperatures, there is one small and thermodynamically unstable black hole, and one large and stable. For $\Lambda R^2=1$, for high temperatures, there is the unstable black hole, and the neutrally stable Nariai universe. Phase transitions can be analyzed. The transitions are between the black hole and hot de Sitter and between Nariai and hot de Sitter. The Buchdahl radius, the radius for collapse, plays an interesting role in the analysis.


title: The 2d Free Boson Minkowski CFT with Asymptotic Charges

abstract: We explicitly construct the states of the 2d free boson on the infinite line with specified asymptotic charges. The Minkowski CFT states derive from (the analytic continuation of) the shrinking limit of Euclidean angular quantization, wherein the superselection sector for fixed asymptotic charges corresponds to specific endpoint operators for the angular quantization on the sphere. When applied to string theory, we obtain a new class of fundamental strings that are neither open nor closed and which we dub "stretched strings". We describe constant-time snapshots of the worldsheets of the different types of stretched strings, and as a special case we find a heuristic picture of Maldacena's "long strings" in the $c=1$ string theory.


title: Age uncertainties of red giants due to cumulative rotational mixing of progenitors calibrated by asteroseismology

abstract: Galactic archaeology largely relies on precise ages of distant evolved stars in the Milky Way. Nowadays, asteroseismology can deliver ages for many red giants observed with high-cadence, high-precision photometric space missions. Our aim is to quantify age uncertainties of slowly-rotating red giants due to the cumulative effect of their fast rotation during core-hydrogen burning. Their rotation in earlier evolutionary phases caused mixing resulting in heavier helium cores and the prolongation of their main sequence. These rotational effects are usually ignored when age-dating red giants, despite our knowledge of fast rotation for stars with $M\ge1.3\,$M$_\odot$. We use a sample of 490 $\gamma$ Doradus pulsators with precise asteroseismic estimates of their internal rotation rate and with luminosity estimates from Gaia. For this sample, which includes stars rotating from nearly 0 to about 60% of the critical rate, we compute the cumulative effect on the age in their post-main sequence evolution caused by rotational mixing on the main sequence. We use stellar model grids with different physical prescriptions mimicking rotational mixing to assess systematic uncertainties on the age. With respect to non-rotating models, the sample of 490 stars, as red giant progenitors, reveals age differences up to 5% by the time they start hydrogen-shell burning when relying on the theory of rotationally induced diffusive mixing as included in the MIST isochrones. Using rotational mixing based on an advective-diffusive approach including meridional circulation leads to an age shift of 20% by the time of the TRGB. Age-dating of red giants is affected by the cumulative effect of rotational mixing during the main sequence. Such rotationally-induced age shifts should be taken into account in addition to other effects if the aim is to perform Galactic archaeological studies at the highest precision. (abridged)


title: Spectroscopic Confirmation of Obscured AGN Populations from Unsupervised Machine Learning

abstract: We present the result of a spectroscopic campaign targeting Active Galactic Nucleus (AGN) candidates selected using a novel unsupervised machine-learning (ML) algorithm trained on optical and mid-infrared (mid-IR) photometry. AGN candidates are chosen without incorporating prior AGN selection criteria and are fainter, redder, and more numerous, $\sim$340 AGN deg$^{-2}$, than comparable photometric and spectroscopic samples. In this work we obtain 178 rest-optical spectra from two candidate ML-identified AGN classes with the Hectospec spectrograph on the MMT Observatory. We find that our first ML-identified group, is dominated by Type I AGNs (85%) with a $<3$% contamination rate from non-AGNs. Our second ML-identified group is comprised mostly of Type II AGNs (65%) with a moderate contamination rate of 15% primarily from star-forming galaxies. Our spectroscopic analyses suggest that the classes recover more obscured AGNs, confirming that ML techniques are effective at recovering large populations of AGNs at high levels of extinction. We demonstrate the efficacy of pairing existing WISE data with large-area and deep optical/near-infrared photometric surveys to select large populations of AGNs and recover obscured SMBH growth. This approach is well suited to upcoming photometric surveys, such as Euclid, Rubin, and Roman.


title: Two-loop anomalous dimensions for small-$R$ jet versus hadronic fragmentation functions

abstract: We study the collinear fragmentation of highly energetic jets defined with a small jet radius. In particular, we investigate how the corresponding fragmentation functions differ from their hadronic counterpart defined in the common $\overline{\rm MS}$ scheme. We find that the anomalous dimensions governing the perturbative evolution of the two fragmentation functions differ starting at the two loop order. We compute for the first time the new anomalous dimensions at two loops and confirm our predictions by comparing the inclusive small-$R$ jet spectrum against a fixed order perturbative calculation at ${\cal O}(\alpha_s^2)$. To investigate the dependence of the anomalous dimension on the kinematic cutoff variable, we study the fragmentation functions of Cambridge jets defined with a transverse momentum cutoff as opposed to an angular cutoff $R$. We further study the evolution of the small-$R$ fragmentation function with an alternative cutoff scale, proportional to $z R$, representing the maximum possible transverse momentum of emissions within a jet. In these cases we find that the two-loop anomalous dimensions coincide with the $\overline{\rm MS}$ DGLAP ones, highlighting a correspondence between the $\overline{\rm MS}$ scheme and a transverse-momentum cutoff.


title: Probability distributions of initial rotation velocities and core-boundary mixing efficiencies of {\gamma} Doradus stars

abstract: The theory the rotational and chemical evolution is incomplete, thereby limiting the accuracy of model-dependent stellar mass and age determinations. The $\gamma$ Doradus pulsators are excellent points of calibration for the current state-of-the-art stellar evolution models, as their gravity modes probe the physical conditions in the deep stellar interior. Yet, individual asteroseismic modelling of these stars is not always possible because of insufficient observed oscillation modes. This paper presents a novel method to derive distributions of the stellar mass, age, core-boundary mixing efficiency and initial rotation rates for $\gamma$ Dor stars. We compute a grid of rotating stellar evolution models covering the entire $\gamma$ Dor instability strip. We then use the observed distributions of the luminosity, effective temperature, buoyancy travel time and near-core rotation frequency of a sample of 539 stars to assign a statistical weight to each of our models. This weight is a measure of how likely the combination of a specific model is. We then compute weighted histograms to derive the most likely distributions of the fundamental stellar properties. We find that the rotation frequency at zero-age main sequence follows a normal distribution, peaking around 25% of the critical Keplerian rotation frequency. The probability-density function for extent of the core-boundary mixing zone, given by a factor $f_{\rm CBM}$ times the local pressure scale height (assuming an exponentially decaying parameterisation) decreases linearly with increasing $f_{\rm CBM}$. Converting the distribution of fractions of critical rotation at the zero-age main sequence to units of d$^{-1}$, we find most F-type stars start the main sequence with a rotation frequency between 0.5 and 2 d$^{-1}$. Regarding the core-boundary mixing efficiency, we find that it is generally weak in this mass regime.


title: Can quantum statistics help distinguish Dirac from Majorana neutrinos?

abstract: Finding out if neutrinos are Dirac or Majorana particles is known to be extremely difficult due to the smallness of neutrino mass and the fact that in the limit $m_\nu=0$ both Dirac and Majorana neutrinos become Weyl particles, i.e. are indistinguishable. There have been suggestions in the literature that in the case of processes with production of a neutrino-antineutrino pair (if neutrinos are Dirac particles) or two neutrinos (if they are of Majorana nature) quantum statistics may be of help. This is because for Majorana neutrinos quantum indistinguishability of identical particles requires the amplitude of the process to be antisymmetrized with respect to the interchange of the final-state neutrinos, whereas no such antisymmetrization must be done for Dirac neutrinos. It has been claimed that the resulting differences between the cross sections for Dirac and Majorana neutrinos persist even for arbitrarily small but not exactly vanishing neutrino mass. We demonstrate that, at least in the framework of the Standard Model, this is not the case. We also give a general proof that within the Standard Model quantum statistics does not help tell Dirac and Majorana neutrinos apart in the limit of negligibly small $m_\nu/E$.


title: Towards Understanding Inductive Bias in Transformers: A View From Infinity

abstract: We study inductive bias in Transformers in the infinitely over-parameterized Gaussian process limit and argue transformers tend to be biased towards more permutation symmetric functions in sequence space. We show that the representation theory of the symmetric group can be used to give quantitative analytical predictions when the dataset is symmetric to permutations between tokens. We present a simplified transformer block and solve the model at the limit, including accurate predictions for the learning curves and network outputs. We show that in common setups, one can derive tight bounds in the form of a scaling law for the learnability as a function of the context length. Finally, we argue WikiText dataset, does indeed possess a degree of permutation symmetry.


title: Three-point Energy Correlators in Hadronic Higgs Decays

abstract: We present the analytic calculation of the leading order three-point energy correlator (EEEC) in hadronic Higgs decays, including both gluon-initiated channel $H\rightarrow g g+X$ and quark-initiated channel $H\rightarrow q\bar q+X$. The phase space integration is evaluated directly using Mandelstam variables $s_{ij}=(p_i+p_j)^2$, and the appearing square roots can be rationalized by either conformal ratios or celestial coordinate variables. Throughout the calculation, we observe the same transcendental function space as in $\mathcal{N}=4$ super Yang-Mills (SYM) theory and $e^+e^-\rightarrow \text{ hadrons}$. Different infrared limits are also explored using the full analytic result, offering the fixed-order data for EEEC factorization and resummation. Given its non-trivial shape dependence, the EEEC presents an excellent opportunity to explore the dynamics of gluon jets originating from the $H \to gg$ decay channel at future lepton colliders.