Statutory article retrieval is the task of automatically retrieving law articles relevant to a legal question. While recent advances in natural language processing have sparked considerable interest in many legal tasks, statutory article retrieval remains primarily untouched due to the scarcity of large-scale and high-quality annotated datasets. To address this bottleneck, we introduce the Belgian Statutory Article Retrieval Dataset (BSARD), which consists of 1,100+ French native legal questions labeled by experienced jurists with relevant articles from a corpus of 22,600+ Belgian law articles. Using BSARD, we benchmark several state-of-the-art retrieval approaches, including lexical and dense architectures, both in zero-shot and supervised setups. We find that fine-tuned dense retrieval models significantly outperform other systems. Our best performing baseline achieves 74.8% R @ 100, which is promising for the feasibility of the task and indicates there is still room for improvement. By the specificity of the domain and addressed task, BSARD presents a unique challenge problem for future research on legal information retrieval. Our dataset and source code are publicly available. Legal issues are an integral part of many people's lives (Ponce et al., 2019). However, the majority of citizens have little to no knowledge about their rights and fundamental legal processes (Balmer et al., 2010). As the Internet has become the primary source of information in response to life problems (Estabrook et al., 2007), people increasingly turn to search engines when faced with a legal issue (Denvir, 2016). Nevertheless, the quality of the search engine's legal help results is currently unsatisfactory, as top results mainly refer people to commercial websites that provide basic information as a way to advertise for-profit services (Hagan and Li, 2020). On average, only one in five persons obtain help from the Internet to clarify or solve their legal issue (Ponce et al., 2019). As a result, many vulnerable citizens who cannot afford a legal expert's costly assistance are left unprotected or even exploited. This barrier to accessing legal information creates a clear imbalance within the legal system, preventing the right to equal access to justice for all. People do not need legal services in and of themselves; they need the ends that legal services can provide. Recent advances in natural language processing (NLP), combined with the increasing amount of digitized textual data in the legal domain, offer new possibilities to bridge the gap between people and the law. For example, legal judgment prediction (Aletras et al., 2016; Luo et al., 2017; Zhong et al., 2018; Hu et al., 2018; Chen et al., 2019) may assist citizens in finding insightful patterns between their case and its outcome. Additionally, legal text summarization (Hachey and Grover, 2006; Bhattacharya et al., 2019) and automated contract review (Harkous et al., 2018; Lippi et al., 2019) may help people clarify long, complex, and ambiguous legal documents. In this work, we focus on statutory article retrieval, which, given a legal question such as  Is it legal to contract a lifetime lease? , aims to return one or several relevant law articles from a body of legal statutes (Kim et al., 2019; Nguyen et al., 2020), as illustrated in Figure 1. A qualified statutory article retrieval system could provide a professional assisting service for unskilled humans and help empower the weaker parties when used for the public interest. Finding relevant statutes to a legal question is a challenging task. Unlike traditional ad-hoc information retrieval (Craswell et al., 2020), statutory article retrieval deals with two types of language: common natural language for the questions and complex legal language for the statutes. This difference in language distribution greatly complicates 6789 Shouldphysicians,surgeons,healthofficers,pharmacists,midwives,andallotherswho,throughtheirstatusorprofession,beinpossessionofinformationconfidedtothemrevealsuchsecrets, they shall be punished with imprisonmentofonetothreeyearsandafineof100to1000eurosoroneofthesepenaltiesonly unless called to testify as a witness in a court of law (or before a parliamentary commission of inquiry) or compelled by a decreeorordertodivulgethesecret. Article458,PenalCode Relevant article What do I risk if I violate professional confidentiality? the retrieval task as it indirectly requires an inherent interpretation system that can translate a natural question from a non-expert to a legal question to be matched against statutes. For skilled legal experts, these interpretations come from their knowledge of a question's domain and their understanding of the legal concepts and processes involved. Nevertheless, an interpretation is rarely unique. Instead, it is the interpreter's subjective belief that gives meaning to the question and, accordingly, an idea of the domains in which the answer can be found. As a result, the same question can yield different paths to the desired outcome depending on its interpretation, making statutory article retrieval a difficult and time-consuming task. Besides, statutory law is not a stack of independent articles to be treated as complete sources of information on their own  unlike news or recipes. Instead, it is a structured and hierarchical collection of legal provisions that have whole meaning only when considered in their overall context, i.e., together with the supplementary information from their neighboring articles, the fields and sub-fields they belong to, and their place in the hierarchy of the law. For instance, the answer to the question  Can I terminate an employment contract?  will most often be found in labor law. However, this is not necessarily true if an employer is contracting a self-employed worker to carry out a specific task, in which case the answer probably lies at the higher level of contract law. This example illustrates the importance of considering the question's context and understanding the hierarchical structure of the law when looking for relevant statutory articles. In order to study whether retrieval models can approximate the efficiency and reliability of legal experts, we need a suitable labeled dataset. However, such datasets are difficult to obtain considering that, although statutory provisions are generally publicly accessible (yet often not in a machine-readable for-mat), the questions posed by citizens are not. This work presents a novel French native expert-annotated statutory article retrieval dataset as its main contribution. Our Belgian Statutory Article Retrieval Dataset (BSARD) consists of more than 1,100 legal questions posed by Belgian citizens and labeled by legal experts with references to relevant articles from a corpus of around 22,600 Belgian law articles. As a second contribution, we establish strong baselines on BSARD by comparing diverse state-of-the-art retrieval approaches from lexical and dense architectures. Our results show that fine-tuned dense retrieval models significantly outperform other approaches yet suggest ample opportunity for improvement. We publicly release our dataset and source code at https: //github.com/maastrichtlawtech/bsard . Due to the increasing digitization of textual legal data, the NLP community has recently introduced more and more datasets to help researchers build reliable models on several legal tasks. For instance, Fawei et al. (2016) introduced a legal question answering (LQA) dataset with 400 multi-choices questions based on the US national bar exam. Similarly, Zhong et al. (2020) released an LQA dataset based on the Chinese bar exam consisting of 26,365 6790 multiple-choice questions, together with a database of evidence that includes 3,382 Chinese legal provisions and the content of the national examination counseling book. Furthermore, Duan et al. (2019) proposed a legal reading comprehension dataset with 52,000 question-answer pairs crafted on the fact descriptions of 10,000 cases from the Supreme People's Court of China. On a different note, Xiao et al. (2018) presented a dataset for legal judgment prediction (LJP) with around 2.68 million Chinese criminal cases annotated with 183 law articles and 202 charges. Likewise, Chalkidis et al. (2019a) introduced an LJP dataset consisting of 11,478 English cases from the European Court of Human Rights labeled with the associated final decision. Meanwhile, Xiao et al. (2019) introduced a dataset for similar case matching with 8,964 triplets of cases published by the Supreme People's Court of China, and Chalkidis et al. (2019b) released a text classification dataset containing 57,000 English EU legislative documents tagged with 4,271 labels from the European Vocabulary. Additionally, Manor and Li (2019) introduced a legal text summarization dataset consisting of 446 sets of contract sections and corresponding reference summaries, and Holzenberger et al. (2020) presented a statutory reasoning dataset based on US tax law. Recently, Hendrycks et al. (2021) proposed a dataset for legal contract review that includes 510 contracts annotated with 41 different clauses for a total of 13,101 annotations. In the same vein, Borchmann et al. (2020) introduced a semantic retrieval dataset for contract discovery with more than 2,500 annotations in around 600 documents. Lastly, the COLIEE Case Law Corpus (Rabelo et al., 2020) is a case law retrieval and entailment dataset that includes 650 base cases from the Federal Court of Canada, each with 200 candidate cases to be identified as relevant to the base case. Regarding statutory article retrieval, the only other publicly available dataset is the COLIEE Statute Law Corpus (Rabelo et al., 2020). It comprises 696 questions from the Japanese legal bar exam labeled with references to relevant articles from the Japanese Civil Code, where both the questions and articles have been translated from Japanese to English. However, this dataset focuses on legal bar exam question answering, which is quite different from legal questions posed by ordinary citizens. While the latter tend to be vague and straightforward, bar exam questions are meant for aspiring lawyers and are thus specific and advanced. Besides, the dataset only contains closed questions (i.e., questions with yes or no answers) and considers almost 30 times fewer law articles than BSARD does. Also, unlike BSARD, the data are not native sentences but instead translated from a foreign language with a completely different legal system. 1 As a result, the translated dataset may not accurately reflect the logic of the original legal system and language. These limitations suggest the need for a novel large-scale citizen-centric native dataset for statutory article retrieval, which is the core contribution of the present work. We create our dataset in four stages: (i) compiling a large corpus of Belgian law articles, (ii) gathering legal questions with references to relevant law articles, (iii) refining these questions, and (iv) matching the references to the corresponding articles from our corpus. Law articles collection. In civil law jurisdictions, a legal code is a type of legislation that purports to exhaustively cover a whole area of law, such as criminal law or tax law, by gathering and restating all the written laws in that area into a unique book. Hence, these books constitute valuable resources to collect many law articles on various subjects. We consider 32 publicly available Belgian codes, as presented in Table 3 of Appendix A. Together with the legal articles, we extract the corresponding headings of the sections in which these articles appear (i.e., book, part, act, chapter, section, and subsection names). These headings provide an overview of each article's subject. As preprocessing, we use regular expressions to clean up the articles of specific wording indicating a change in part of the article by a past law (e.g., nested brackets, superscripts, or footnotes). Additionally, we identify and remove the articles repealed by past laws but still present in the codes. Eventually, we end up with a corpus C = { a 1 ,    , a N } 1 Japan is a civil law country that relies predominantly on the rules written down in statutes, whereas most Englishspeaking countries (e.g., US, UK, Canada, and Australia) have a common law system that relies predominantly on past judicial decisions, known as precedents. Questions collection. We partner with Droits Quotidiens (DQ), 2 a Belgian organization whose mission is to clarify the law for laypeople. Each year, DQ receives and collects around 4,000 emails from Belgian citizens asking for advice on a personal legal issue. Thanks to these emails, its team of six experienced jurists keeps abreast of Bel-gium's most common legal issues and addresses them as comprehensively as possible on its website. Each jurist is an expert in a specific field (e.g., family, housing, or work) and is responsible for answering all questions related to that field. Given their qualifications and years of experience in providing legal advice in their respective fields, the experts can be considered competent enough to always (eventually) retrieve the correct articles to a given question. In practice, their legal clarification process consists of four steps. First, they identify the most frequently asked questions on a common legal issue. Then, they define a new anonymized model question on that issue expressed in natural language terms, i.e., as close as possible as if a layperson had asked it. Next, they search the Belgian law for articles that help answer the model question and reference them. Finally, they answer the question using the retrieved relevant articles in a way a layperson can understand. These model questions, legal references, and answers are further categorized before being posted on DQ's website (e.g., the question  What is the seizure of goods?  is tagged under the  Money  Debt recovery  category). With their consent, we collect more than 3,200 model questions together with their references to relevant law articles and categorization tags. Assuming it takes a jurist between 5 to 20 minutes to find the relevant articles to a given question and categorize the latter. An estimate of the pecuniary value of those labeled questions is over C105,000  3,200 questions, each requiring 10 minutes to label, assuming a rate of C200 per hour. Questions refinement. We find that around one-third of the collected questions are duplicates. However, these duplicated questions come with different categorization tags, some of which providing additional context that can be used to refine the questions. For example, the question  Should I 2 https://droitsquotidiens.be/ install fire detectors?  appears four times in total, under the following tags:  Housing  Rent  I am a { tenant , landlord }  In { Wallonia , Brussels } . We distinguish between the tags with one or a few words indicating a question subject (e.g.,  housing  and  rent ) and those that provide context about a personal situation or location as short descriptive sentences (e.g.,  I am tenant in Brussels. ). If any, we append the contextual sentence tags in front of the questions, which solves most of the duplicates problem and improves the overall quality of the questions by making them more specific. Questions filtering. The questions collected are annotated with plain text references to relevant law articles (e.g.,  Article 8 of the Civil Code ). We use regular expressions to parse these references and match them to the corresponding articles from our corpus. First, we filter out questions whose references are not articles (e.g., an entire decree or order). Then, we remove questions with references to legal acts other than codes of law (e.g., decrees, directives, or ordinances). Next, we ignore questions with references to codes other than those we initially considered. We eventually end up with 1,108 questions, each carefully labeled with the ids of the corresponding relevant law articles from our corpus. Finally, we split the dataset into training/test sets with 886 and 222 questions, respectively. To provide more insight, we describe quantitative and qualitative observations about BSARD. Specifically, we explore (i) the diversity in questions and articles, (ii) the relationship between questions and their relevant articles, and (iii) the type of reasoning required to retrieve relevant articles. Diversity. The 22,633 law articles that constitute our corpus have been collected from 32 Belgian codes covering a large number of legal topics, as presented in Table 3 of Appendix A. The articles have a median length of 77 words, but 142 articles exceed 1,000 words (the lengthiest one being up to 5,790 words), as illustrated in Figure 2b. These long articles are mostly general provisions , i.e., articles that appear at the beginning of a code and define many terms and concepts later mentioned in the code. The questions are between 5 and 44 words long, with a median of 14 words, as shown in Figure 2a. They cover a wide range of topics, with around 85% of them being either about family, 6792 General topic Percentage Subtopics Example Family 30.6% Marriage, parentage, divorce, etc. housing, money, or justice, while the remaining 15% concern either social security, foreigners, or work, as described in Table 1. Question-article relationship. Questions might have one or several relevant legal articles. Overall, 75% of the questions have less than five relevant articles, 18% have between 5 and 20, and the remaining 7% have more than 20 with a maximum of 109, as seen in Figure 2c. The latter often have complex and indirect answers that demand extensive reasoning over a whole code section, which explains these large numbers of relevant articles. Furthermore, an article deemed relevant to one question might also be for others. Therefore, we calculate for each unique article deemed relevant to at least one question the total number of times it is cited as a legal reference across all questions. As a result, we find that the median number of citations for those articles is 2, and less than 25% of them are cited more than five times, as illustrated in Figure 2d. Hence, out of the 22,633 articles, only 1,612 are referred to as relevant to at least one question in the dataset, and around 80% of these 1,612 articles come from either the Civil Code, Judicial Code, Criminal Investigation Code, or Penal Code. Meanwhile, 18 out of the 32 codes have less than five articles mentioned as relevant to at least one question, which can be explained by the fact that those codes focus less on individuals and their concerns. Formally speaking, a statutory article retrieval system R : ( q, C )  F is a function that takes as input a question q along with a corpus of law articles C , and returns a much smaller filter set F  C of the supposedly relevant articles, ranked by decreasing order of relevance. For a fixed k = |F|  |C| , the retriever can be evaluated in isolation with multiple rank-based metrics (see Section 5.1). The following section describes the retrieval models we use as a benchmark for the task. Traditionally, lexical approaches have been the de facto standard for textual information retrieval due to their robustness and efficiency. Given a query q and an article a , a lexical model assigns to the pair ( q, a ) a score s L : ( q, a )  R + by computing the sum, over the query terms, of the weights of each query term t  q in the article, i.e., s L ( q, a ) = (cid:88) t  q w ( t, a ) . where the term frequency tf is the number of occurrences of term t in article a , and the document frequency df is the number of articles within the corpus that contain term t . Then, we experiment with the BM25 weighting formula (Robertson et al., 1994), defined as w ( t, a ) = tf( t, a )  ( k 1 +1) tf( t, a )+ k 1  (cid:16) 1  b + b  | a | avgal (cid:17)  log |C|  df( t ) + 0 . 5 df( t ) + 0 . 5 , (3) where k 1  R + and b  [0 , 1] are constant parameters to be fixed, | a | is the article length, and avgal is the average article length in the collection. During inference, we compute a score for each article in corpus C and return the k articles with the highest scores as the topk most relevant results to the input query. Lexical approaches suffer from the lexical gap problem (Berger et al., 2000) and can only retrieve articles containing keywords present in the query. To overcome this limitation, recent work (Lee et al., 2019; Karpukhin et al., 2020; Xiong et al., 2021) relies on neural-based architectures to capture semantic relationships between the query and documents. The most commonly used approach is based on a bi-encoder model (Gillick et al., 2018) that maps queries and documents into dense vector representations. Formally, a dense retriever calculates a relevance score s D : ( q, a )  R + between question q and article a by the similarity of their respective embeddings h q , h a  R d , i.e., s D ( q, a ) = sim ( h q , h a ) , (4) where sim : R d  R d  R is a similarity function such as dot product or cosine similarity. Typically, these embeddings result from a pooling operation on the output representations of a word embedding model: h q = pool ( f ( q ;  1 )) , and h a = pool ( f ( a ;  2 )) , (5) where model f (  ;  i ) : W n  R n  d with parameters  i maps an input text sequence of n terms from vocabulary W to d -dimensional real-valued word vectors. The pooling operation pool : R n  d  R d uses the output word embeddings to distill a global representation for the text passage  using either mean, max, or [CLS] pooling. Note that the bi-encoder architecture comes with two flavors: (i) siamese (Reimers and Gurevych, 2019; Xiong et al., 2021), which uses a unique word embedding model (i.e.,  1 =  2 ) that maps the query and article together in a shared dense vector space, and (ii) two-tower (Yang et al., 2020; Karpukhin et al., 2020), which use two independent word embedding models that encode the query and article separately into different embedding spaces. During inference, the articles are pre-encoded offline, and their representations are stored in an index structure. Then, given an input query, an exact search is performed by computing the similarities between the query representation and all pre-encoded article representations. The resulting scores are used to rank the articles such that the k articles that have the highest similarities with the query are returned as the topk results. First, we study the effectiveness of siamese bi-encoders in a zero-shot evaluation setup, i.e., pre-trained word embedding models are applied out-of-the-box without any additional fine-tuning. We experiment with two types of widely-used word embedding models: (i) models that learned contextindependent word representations, namely word2vec (Mikolov et al., 2013a,b) and fastText (Bojanowski et al., 2017), and (ii) models that learned contextdependent word embeddings, namely RoBERTa (Liu et al., 2019). RoBERTa can process texts up to a maximum input length of 512 tokens. Although alternative models exist to alleviate this limitation (Beltagy et al., 2020; Ainslie et al., 2020), they have all been trained on English text, and there are no French equivalents available yet. Therefore, we use a simple workaround that splits the text into overlapping chunks and passes each chunk in turn to the embedding model. To form the chunks, we consider contiguous text sequences of 200 tokens with an overlap of 20 tokens between consecutive chunks. For all zero-shot models, we use mean pooling on all word embeddings of the passage to extract a global representation for the latter and cosine similarity to score passage representations. Thereafter, we train our own siamese and two-tower RoBERTa-based bi-encoder models on BSARD. Let D = { q i , a + i } Ni =1 be the training data where each of the N instances consists of a query q i associated with a relevant (positive) article a + i . Using in-batch negatives (Chen et al., 2017; Henderson et al., 2017), we can create a training set T = { q i , a + i , A  i } Ni =1 where A  i is a set of negative articles for question q i constructed by considering the articles paired with the other questions from the same mini-batch. For each training instance, we contrastively optimize the negative log-likelihood of each positive article against their negative articles, i.e., L (cid:0) q i , a + i , A  i (cid:1) =  log exp (cid:0) s D ( q i , a + i ) / (cid:1) (cid:80) a A  i { a + i } exp ( s D ( q i , a ) / ) , (6) where  > 0 is a temperature parameter to be set. This contrastive loss allows learning embedding functions such that relevant question-article pairs will have a higher score than irrelevant ones. To deal with articles longer than 512 tokens, we use the same workaround as in the zero-shot evaluation and split the long sequences into overlapping chunks of 200 tokens with a window size of 20. However, this time, we limit the size of the articles to the first 1,000 words due to limited GPU memory. Although not ideal, doing so remains reasonable given that only 0.6% of the articles in our corpus have more than 1,000 words, as mentioned in Section 3.2. Each chunk is prefixed by the [CLS] token, and we extract a global representation for the whole article by averaging the output [CLS] token embeddings of the different chunks. Here, we use the dot product to compute similarities as it gives slightly better results than cosine. Metrics. We use three standard information retrieval metrics (Manning et al., 2008) to evaluate performance, namely the (macro-averaged) recall @ k (R @ k ), mean average precision @ k (MAP @ k ), and mean reciprocal rank @ k (MRR @ k ). Appendix B gives a detailed description of these metrics in the context of statutory article retrieval. We deliberately omit to report the precision @ k given that questions have a variable number of relevant articles (see Figure 2c), which makes it senseless to report it at a fixed k  questions with r relevant articles will always have P @ k < 1 if k > r . For the same reason, k should be large enough for the recall @ k . Hence, we use k  { 100 , 200 , 500 } for our evaluation. French word embedding models. Our focus is on a non-English dataset, so we experiment with French variants of the models mentioned above. Specifically, we use a 500-dimensional skip-gram word2vec model pre-trained on a crawled French corpus (Fauconnier, 2015), a 300-dimensional CBOW fastText model pre-trained on French Web data (Grave et al., 2018), and a French RoBERTa model, namely CamemBERT (Martin et al., 2020), pre-trained on 147GB of French web pages filtered from Common Crawl. 3 Hyper-parameters & schedule. For BM25, we optimize the parameters on BSARD training set and find k 1 = 1 . 0 and b = 0 . 6 to perform best. Regarding the bi-encoder models, we optimize the contrastive loss using a batch size of 22 question-article pairs and a temperature of 0.05 for 100 epochs, which is approximately 20,500 steps. We use AdamW (Loshchilov and Hutter, 2019) with an initial learning rate of 2e-5,  1 = 0 . 9 ,  2 = 0 . 999 , weight decay of 0.01, learning rate warm up over the first 500 steps, and linear decay of the learning rate. Training is performed on a single Tesla V100 GPU with 32 GBs of memory and evaluation on a server with a dual 20 core Intel(R) Xeon(R) E5-2698 v4 CPU @ 2.20GHz and 512 GBs of RAM. In Table 2, we report the retrieval performance of our models on the BSARD test set. Overall, the trained bi-encoder models significantly outperform all the other baselines. The two-tower model improves over its siamese variant on recall @ 100 but performs similarly on the other metrics. Although BM25 underperforms the trained bi-encoders significantly, its performance indicates that it is still a strong baseline for domain-specific retrieval. These results are consistent with those obtained on other in-domain datasets (Thakur et al., 2021). Regarding the zero-shot evaluation of siamese bi-encoder models, we find that directly using the embeddings of a pre-trained CamemBERT model without optimizing for the IR task gives poor results. Reimers and Gurevych (2019) noted similar findings for the task of semantic textual similarity. Furthermore, we observe that the word2vec-based bi-encoder significantly outperforms the fastText and BERT-based models, suggesting that pre-trained word-level embeddings are more appropriate for the task than character-level or subword-level embeddings when used out of the box. Although promising, these results suggest ample opportunity for improvement compared to a skilled legal expert who can eventually retrieve all relevant articles to any question and thus get perfect scores. As our dataset aims to give researchers a well-defined benchmark to evaluate existing and future legal information retrieval models, certain limitations need to be borne in mind to avoid drawing erroneous conclusions. First, the corpus of articles is limited to those collected from the 32 Belgian codes described in Table 3 of Appendix A, which does not cover the entire Belgian law as thousands of articles from decrees, directives, and ordinances are missing. During the dataset construction, all references to these uncollected articles are ignored, which causes some questions to end up with only a fraction of their initial number of relevant articles. This information loss implies that the answer contained in the remaining relevant articles might be incomplete, although it is still appropriate. For instance, the question  Can I evict my tenants if they make too much noise?  might not have a detailed answer within the statutory law that quantifies a specific noise threshold at which eviction is allowed. Instead, the landlord should probably rely more on case law and find precedents similar to their current situation (e.g., the tenant makes two parties a week until 2 am). Hence, some questions are better suited than others to the statutory article retrieval task, and the domain of the less suitable ones remains to be determined. In addition to helping advance the state-of-the-art in retrieving statutes relevant to a legal question, BSARD-based models could improve the efficiency of the legal information retrieval process in the context of legal research, therefore enabling researchers to devote themselves to more thoughtful parts of their research. Furthermore, BSARD can become a starting point of new open-source legal information search tools so that the socially weaker parties to disputes can benefit from a free professional assisting service. However, there are risks that the dataset will not be used exclusively for the public interest but perhaps also for profit as part of proprietary search tools developed by companies. Since this would reinforce rather than solve the problem of access to legal information and justice for all, we decided to distribute BSARD under a license with a noncommercial clause. Other potential negative societal impacts could involve using models trained on BSARD to misuse or find gaps within the governmental laws or use the latter not to defend oneself but to deliberately damage people or companies instead. Of course, we discourage anyone from developing models that aim to perform the latter actions. In this paper, we present the Belgian Statutory Article Retrieval Dataset (BSARD), a citizen-centric French native dataset for statutory article retrieval. Within a larger effort to bridge the gap between people and the law, BSARD provides a means of evaluating and developing models capable of retrieving law articles relevant to a legal question posed by a layperson. We benchmark several strong information retrieval baselines that show promise for the feasibility of the task yet indicate room for improvement. In the future, we plan to build retrieval models that can handle lengthy statutory articles and inherently exploit the hierarchy of the law. In closing, we hope that our work sparks interest in developing practical and reliable statutory article retrieval models to help improve access to justice for all. This research is partially supported by the Sector Plan Digital Legal Studies of the Dutch Ministry of Education, Culture, and Science. In addition, this research was made possible, in part, using the Data Science Research Infrastructure (DSRI) hosted at Maastricht University.
abstain abstain abstain method result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain objective abstain objective result other other abstain other other other other other method other other other other other other other other other other other method abstain abstain abstain abstain abstain abstain abstain method abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain other method abstain abstain abstain method abstain method method abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain other abstain method abstain abstain abstain abstain abstain abstain other other abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method method method method abstain method method abstain method abstain abstain method abstain abstain method abstain method abstain method abstain method method abstain method abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain result abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain result method objective other other