text:
\section*{Introduction}
Language is the `coin of the realm' in the legal domain. Not only do legal institutions and actors produce, consume, and interpret massive volumes of text,\cite{chalkidis2022lexglue} but virtually every legal process involves either the production or consumption of documents. Careful drafting of documents and the analysis and interpretation of language are among the core activities undertaken by judges, regulators, legislators, and lawyers. Participants in the world's legal systems ``continuously author legal texts such as statutes, regulations, judicial decisions, contracts, patents, briefs, memos, and other related materials.''\cite{coupette2021measuring} Taken together, legal systems output large volumes of documents and these documents are often complex.  Indeed, the ``language of law" has proven to be so challenging that many laypersons describe legal documents and arguments using terms such as `legalese', `legal jargon' or `legal gobbledygook.'
\\
\\
 The complexity of the law\cite{ruhl2017harnessing}\textsuperscript{,} \cite{bommarito2017measuring}\textsuperscript{,}\cite{katz2020complex} is not just a scientific phenomenon; it has real consequences for many individuals and organizations.\cite{ruhl2015measuring}\textsuperscript{,}\cite{staudt2008all} In part due to this complexity, legal systems have struggled to assist with ``the quantity, quality, and accessibility of legal services demanded by society.''\cite{bommarito2022gpt} Yet, despite this underlying and obvious need for improvement in the delivery of justice,\cite{rhode2004access}\textsuperscript{,}\cite{susskind2019online}\textsuperscript{,}\cite{sandefur2020assessing}\textsuperscript{,}\cite{prescott2017improving} there have been many barriers which have prevented the emergence of scalable solutions to meet various legal needs. These barriers include the culture of law (including lawyers, judges, and legal educators)\cite{susskind2017tomorrow}\textsuperscript{,}\cite{barton2017rebooting} as well as the regulation of the profession. \cite{kobayashi2011law}\textsuperscript{,}\cite{hadfield2014cost}\textsuperscript{,}\cite{barton2018access}  Yet, the primary \textit{technical} challenge limiting transformative technological solutions within the legal sphere is the complex nature of legal language itself.
\\
\\
 Simply put, the task of training machines to ``understand'' legal language has proven to be non-trivial. Notwithstanding the challenge, there has understandably been great interest in exploring the possibility of machines as a force multiplier for helping process complex legal texts. Indeed, both scholars and commercial enterprises have explored the applicability of Natural Language Processing (NLP) technologies for use within the field of law (Legal NLP). In the academic realm, empirical legal studies increasingly rely on a variety of methods from computer science to help support analysis.\cite{fagan2020natural}\textsuperscript{,} \cite{livermore2019law}\textsuperscript{,}\cite{kolt2022predicting}  In the commercial sphere, there also have been attempts to embed Legal NLP modules into a number of applications in legal practice,\cite{bommarito2021lexnlp}\textsuperscript{,}\cite{dale2019law}\textsuperscript{,}\cite{engstrom2020legal} from research tools and litigation outcome prediction to drafting support and compliance risk assessment.  Overall, despite some laudable attempts, the performance of many commercial applications has at times been undermined by the inability to consistently process legal language in a high-fidelity manner.
\\
\\
Meanwhile in the more general technical literature, the past decade has witnessed major gains in the quality and performance of language models.  Building upon foundational advances in neural network research, \cite{Rumelhart1986}\textsuperscript{,}\cite{LeCun2015} the broader field of NLP has been reshaped in this period.  While early neural NLP papers were built upon word embeddings,\cite{mikolov2013distributed}\textsuperscript{,}\cite{Pennington2014glove}\textsuperscript{,}\cite{peters2018deep} the latest wave of LLMs is being built upon the transformer architecture.\cite{vaswani2017attention}  Among other things, the transformer architecture allows for the clever manipulation of the attention mechanism so that training tasks can be scaled through more effective parallelization.\cite{tay2022efficient}  Notwithstanding some critiques,\cite{bender2021dangers} successive waves of increasingly large transformer-based large language models (LLMs) have delivered some truly remarkable results.\cite{kenton2019bert}\textsuperscript{,}\cite{brown2020language}\textsuperscript{,}\cite{zaheer2020big}\textsuperscript{,} \cite{scao2022bloom}\textsuperscript{,}\cite{thoppilan2022lamda}
\\
\\
Despite sometimes being characterized as general models, it is still an open question as to how much uptake or utility such core developments in NLP might offer when directed at complex domain-specific problems.  While general models have shown real progress on legal tasks in the zero shot context,\cite{chalkidis2022lexglue}\textsuperscript{,} \cite{bommarito2022gpt}\textsuperscript{,}\cite{zheng2021does}\textsuperscript{,}\cite{chalkidis2021multieurlex}\textsuperscript{,}\cite{nay2023large}\textsuperscript{,}\cite{bommarito2023gpt} there are still strong reasons\cite{Huang2022}\textsuperscript{,}\cite{wu2022promptchainer}\textsuperscript{,}\cite{zhou2022large} to believe that some combination of domain-specific pre-training, prompt engineering, prompt composition or chaining, hyper-parameter optimization, and other model tuning efforts will yield improved results in many substantive use cases. In other words, general NLP models will likely not eclipse the performance of an otherwise equally-sized large language model that has been well trained on the legal domain.  That said, general models with enormous scale may very well outperform lower scale domain models.

\section*{Building a Corpus of Legal NLP Papers}
Fueled by the wide scale expansion of digitized legal texts, the prospect for deploying cutting-edge NLP techniques within the legal domain has become increasingly possible.  To support those efforts, while also providing a roadmap to various interdisciplinary scholars, we thought it to be useful to summarize emergent trends in the field.  Although there have been efforts to characterize some developments in the field, including analysis of Legal NLP alone \cite{fagan2020natural}\textsuperscript{,}\cite{zhong2020does} as well as discussions situated in the broader context of legal informatics,\cite{katz2021legal}\textsuperscript{,} \cite{ashley2017artificial} we  believe a more holistic treatment of the current state of Legal NLP is justified.
\\
\\
Taking the past decade as our window of analysis, we constructed a corpus of nearly all published Legal NLP papers. To build the corpus, we began by reviewing the proceedings of both general NLP conferences (e.g. ACL, NAACL, EMNLP, EACL, etc.) as well as several speciality Legal NLP gatherings (NLLP, MLLD, Jurix, ICAIL, etc.).  This initial screen yielded a large number of results.  Next, we performed iterative queries on both major search engines and publication databases. Finally, we undertook a form of `snowball sampling' by manually traversing the citation graph of key publications to scour for additional NLP \& Law papers not otherwise identified using the approach above. Absent extraordinary circumstances, we restricted our corpus to include only peer-reviewed scientific journals, technical conference proceedings, and otherwise technically-oriented pre-prints (\textit{e.g}. SSRN, arXiv).  In other words, we generally excluded essays, commentaries, blog posts, social critiques, or otherwise non-technical and/or non-scientific publications.
\\
\\
Law as an intellectual domain is somewhat amorphous and thus some qualitative judgement is required to adjudicate its boundary.  Therefore, care was required to determine whether a given paper is or is not `Legal NLP.'  As an illustrative example, there are many papers that apply NLP to analyze documents related to financial instruments. While the difference between these financial documents and legal documents often approaches zero, the focus and audience of these papers is typically much different.  Therefore, we excluded these sort of efforts due to their limited nexus and connection to the legal domain.  Our guiding principle was to include any technically-inclined paper whose target audience or target user was an individual working in the field of law.
\\
\\
Notwithstanding the aforementioned caveats, we believe we have acquired nearly every paper meeting these criteria published over the past decade. Figure \ref{fig:1} provides a time series representation of the volume of papers published over the past decade.  We observe a significant increase in the total yearly volume of publications with only a slight diminution in 2020 (which is perhaps attributable to disruptive impacts of the early period of the COVID-19 pandemic).

\begin{figure}[t]
\centering
\includegraphics[width=11cm]{figure1.pdf}
\caption{Number of Legal NLP Papers over Time}\label{fig:1}
\end{figure}

\section*{The Engineering Perspective on Legal NLP Tasks}
Law regulates all sectors of society.  As such, the topic space covered therein is quite vast. Relevant topics include constitutional law, environmental law, intellectual property, labor law, corporate law, immigration law, criminal law, tax law, family law, and maritime law.  Within each of these substantive areas, there are many specific tasks which lawyers must undertake. As reflected in various legal task schemas (\textit{e.g.} UTBMS, SALI), legal workstreams can be decomposed into their constituent subtasks.\cite{bartolo2019pre}\textsuperscript{,}\cite{constantinou2020detecting} For example, within a given legal matter, a legal professional might review documents, draft documents, conduct research, give arguments in court, negotiate with counter parties, etc. This diversity in tasks is reflected across our collection of more than six hundred papers, where scholars generally seek to apply Legal NLP techniques to augment, assist, or replace the lawyers as the sole available resource to complete a given task.

\begin{table}[hbt!]
\centering
\begin{tabular}{|l|l|l|}
\hline
  & \multicolumn{1}{c|}{\textbf{Task}} & \textbf{Examples / Description}                                                                                \\ \hline
1 & \textbf{Machine Summarization}     & Abstractive/Extractive Summaries of Legal Documents                                                                                \\ \hline2 & \textbf{Pre-Processing}            & Annotation, Anonymization, Translation                                                                                \\ \hline
3 & \textbf{Classification}            & \begin{tabular}[c]{@{}l@{}}Outcome Prediction, Legal Area Classification, Topic Modeling
\end{tabular} \\ \hline
4 & \textbf{Information Retrieval}     & Legal Question Answering, Document Similarity, Document Retrieval                                                                                \\ \hline
5 & \textbf{Information Extraction}    & Labeling, Text Extraction, Event Extraction                                                                                \\ \hline
6 & \textbf{Text Generation}           & Automated Drafting of Legal Documents                                                                                \\ \hline
7 & \textbf{Resources}                 & Taxonomies, Ontologies, Datasets, Code Libraries                                                                                \\ \hline
\end{tabular}
\caption{\label{tab}A Taxonomy of Engineering Tasks in Legal NLP}
\end{table}

\begin{figure}[t]
\centering
\includegraphics[width=18cm]{figure2.pdf}\\
\caption{Legal NLP Tasks over Time}\label{fig:2}
\end{figure}

\noindent
In addition to the more lawyer-centric perspective of legal work, the space of NLP \& Law tasks can be considered through the lens of engineering.  Relevant engineering tasks could include machine summarization, translation, classification, retrieval, etc. Of course, these perspectives will often intersect. Consider a paper which explores automated extraction of keywords from a patent claim in order to support the search for prior art (i.e. other relevant patents).\cite{rossi2018query}  This paper can be characterized as a patent paper from a substantive law perspective and as an information retrieval type paper from an engineering perspective. Recognizing this dualism attaches to the vast majority of papers in our corpus, we will---for the purposes of our analysis---privilege the engineering perspective on Legal NLP tasks.  While certainly not the only way one could subdivide the space, Table \ref{tab} offers an engineering-centric taxonomy of Legal NLP tasks.
\\
\\
Leveraging our taxonomy, we qualitatively reviewed each of the papers in our corpus and determined the relevant engineering category for the work contained therein. While most papers fit squarely within one particular category, there were some papers which combined two or more engineering tasks. Figure \ref{fig:2} reflects the temporal distribution of papers by engineering task.
\\
\\
Subdividing the activity by categories yields interesting results: The general trend---upwards and to the right---is carried through most but not all categories. However, both Summarization and Classification show a change of pace around 2018 or 2019. Information Retrevial, on the other hand, shows fairly stable interest over time. Both Resources and Pre-Processing papers increase over time with the general stable increase of papers.
\\
\\
Some applications, such as Machine Summarization and Text Generation, have seen little activity compared to the more popular fields of Classification and Information Extraction. Given the current interest in text generation through popular exposure to applications such as ChatGPT, we expect a marked increase in the volume of Text Generation papers in the decade to come.

\section*{The Evolution of Methods in Legal NLP}
The ability to work with and process language has long been an interest for scientific researchers.  Indeed, arguably the most famous benchmark in the history of Artificial Intelligence, the Turing Test, involves a  conversational interaction between a human and a computational agent.\cite{turing1950}  The quest to fulfill the promises and goals of the field has taken scholars in many different directions.  As such, NLP has experienced several waves of methodological innovation.
\\
\\
Early work in NLP can be traced to various rules-based systems which were either proposed or implemented.\cite{chomsky1957syntactic}\textsuperscript{,} \cite{schank1973margie}\textsuperscript{,}\cite{lehnert1977conceptual}  The advent of Moore's Law\cite{shalf2020future} and the vast decline in the cost of data storage,\cite{gupta2014economic} taken together with increasingly clever algorithmic methods, saw the field of NLP transformed from its rules-based AI origins into a data-driven field.  This statistical turn, which began in the 1990's, has more recently given way to the `neural era' within the general field of NLP.\cite{zhou2020progress}
\\
\\
Legal AI and Legal NLP is a long-standing field with papers tracing back many decades.\cite{governatori2022thirty}  Over time the field has seen significant methodological innovation.  Consider a recent review of the Legal AI papers (authored by several leading scholars in the field) which discusses the increasing use of sophisticated methods. ``[L]arge convolutional networks trained on graphics processors achieved breakthrough performance in computer vision [whereby] neural models quickly became the dominant technology … This development eventually reached the AI and Law community and led to a surge in the use of such models for the analysis of legal text from around 2017 onwards."\cite{governatori2022thirty}  Although our own qualitative review of our corpus supports this perspective, we thought it might be useful to empirically analyze this proposition.
\\

\begin{figure}[htbp]
    \centering
    \includegraphics[width=13cm]{figure4.pdf}\\
    \caption{Distribution of Phrases by Paper}
    \label{fig:4}
\end{figure}

\noindent Working with the full text of the more than six hundred papers in our Legal NLP corpus, we selected a set of key words which we believed could be tracked in order to understand the nature of methods contained within each paper.  If the ``neural agenda" were indeed ascendant, we would expect to observe an increase in relevant phrases over the most recent years.  Having collected a full text copy of nearly every paper within our corpus,\footnote{As of the release of this pre-print, we have collected full text copies of over 99.5\% of the papers.} we pre-processed each manuscript and extracted the plaintext from each document.  While many more sophisticated approaches could be selected, we sought to simply track the emergence of key phrases which were indicative of `neural turn' in Legal NLP.
\\
\\
Figure \ref{fig:4} offers the distribution of key phrases by paper count.  While phrases such as `neural' are quite common across the corpus, we recognize this alone could be unrelated to the proposition we seek to evaluate.  For example, this could be the byproduct of references to much earlier periods when alternative forms of neural networks were popular.\cite{anderson2000talking} Therefore, we included many other key phrases which are associated with the modern neural NLP methods (\textit{e.g.} embedding, word2vec, LSTM, BERT, etc.).  Figure \ref{fig:5} provides a normalized time series of these phrases on a yearly basis.  While many of these phrases unsurprisingly peak in 2022, we do observe that some first-wave embedding-based approaches (\textit{e.g.} word2vec, doc2vec, etc.) appear to have peaked during the 2018-2021 window.

\begin{figure}[htbp]
    \centering
    \includegraphics[width=14cm]{figure5b.pdf}\\
    \caption{Relative Rate of Term Usage over Time.  Normalization is per-term relative to the maximum annual rate of mentioning papers.}
    \label{fig:5}
\end{figure}

\section*{The Diversity of Languages}
Deciding on the language of the data is a foundational decision at the core of each paper. The English language is historically well-positioned in this regard, as it is the \textit{de facto} lingua franca of both computing and international (business) law.  For this reason, we expected English to dominate the distribution of languages in our papers, with other world languages such as Chinese, German and French to be the nearest contenders.
\\
\begin{figure}[htbp]
    \centering
    \includegraphics[width=12cm]{figure3.pdf}\\
    \caption{Temporal Distribution of the Most Popular Legal NLP Languages as a Function of Time}
    \label{fig:3}
\end{figure}

\noindent This hypothesis was mostly proven correct in our evaluation: The most popular language is---by far---English (56\%). The next most important language is Chinese. Other commonly occurring languages include Japanese, French, and German. In the case of Japanese and German, this may be partially explained by data availability: For Japanese, there are widely-used corpora of the Japanese National Pension Act and other statutes that are used in a number of papers surveyed, which might have an influence on the NLP community there. For the German language, there is ample data available, both at the national level\footnote{\textit{e.g.}, through offerings such as \href{https://www.gesetze-im-internet.de/}{``Gesetze im Internet''} or \href{https://www.rechtsprechung-im-internet.de/}{``Rechtsprechung im Internet''} and similar offerings in Austria and the German-speaking Swiss cantons.} as well as at the supranational level through the European Union.
\\
\\
Figure \ref{fig:3} offers a stack plot of the time series of papers by language.  Observing the distribution of the five most common languages over time per year, the proportion of English-language papers remains roughly constant, while papers analyzing Chinese-language corpora increase substantially.  Additionally, Figure \ref{fig:3} reveals that the overall diversity of languages has increased, with the sum of less common languages increasing substantially over time (cf. the ``other'' band in Figure \ref{fig:3}).  As a word of limitation, it is important to note that we only surveyed papers that were written in the English language.  Therefore, given that there is likely significant scholarship published only in languages such as Chinese, German, French, Spanish, etc. the results offered in Figure \ref{fig:3} should be considered as lower bound estimates for linguistic diversity.

\section*{Reproducibility and Data Availability}
Reproducibility is an increasing concern of not only the NLP community but also the broader scientific community. \cite{munafo2017manifesto}\textsuperscript{,} \cite{ivie2018reproducibility}  Beyond the important task of verifying existing results, transparent and replicable outputs can help accelerate the pace of additive innovation.
\\
\\
For each of the more than six hundred papers, we comprehensively evaluated the nature of replication resources made available by the respective authors.  Of course, in judging the reproducibility of a given paper, many different aspects can be considered including the availability of datasets, corpora, models, proper documentation, etc. For ease, we categorize each the papers in our corpus into one of three classes. \textit{Class I} contains those papers for which either no proper links to the resources are mentioned or the given link does not work, or no links are mentioned at all in the paper.  For this class of papers, it would be quite difficult to reproduce the result only based upon the information provided by the authors. \textit{Class II} papers provide partial resources to support replication / implementation.  The nature of the incomplete element varies.  Some papers provide only data sources, whereas, for some papers, only source codes are available, often with little or no documentation. Therefore, the frameworks or the results presented in this class of papers can not be reproduced straightforwardly, without material assistance from the authors. By contrast, \textit{Class III} papers offer a well-organized repository of resources (both code and data) with proper documentation.
\\
\\
The three classes of papers from a reproducibility standpoint are summarized as follows:

\begin{displayquote}
(1) \textbf{Class I} - No Data or Code is Available\\
(2) \textbf{Class II} - Partial or Incomplete Data and/or Code Availability\\
(3) \textbf{Class III} - Data and Code are Available in an Organized Repository
\end{displayquote}

\noindent
We analyze the availability of resources from two different perspectives. To start in Figure \ref{fig:6}, we investigate whether with time, Legal NLP authors have become more inclined to make resources publicly available. The statistics for the last decade (2013-2022) are quite promising.  We observe that the percentage of \textit{Class I} papers has decreased significantly from 92.59\% (2013) to 48.70\% (2022), whereas the percentage of \textit{Class III} papers increased noticeably from 3.70\% (2013) to 39.13\% (2022). These statistics highlight an increased commitment to reproducibility within the Legal NLP community.
\\

\begin{figure}[t]
\centering
\includegraphics[width=11cm]{figure6.pdf}
\caption{Replication Material Availability as a Function of Time}\label{fig:6}
\end{figure}

\noindent
 We next consider to what extent the language of legal data correlates with the underlying availability of resources. The observations for some of the most researched languages (English, Chinese, German, Japanese \& French) are presented in Figure \ref{fig:7}. Even though the number of research papers analyzing English legal data is the highest by a considerable margin, the percentage of \textit{Class I} papers for English is also quite high (59.59\%), and the percentage of \textit{Class III} papers is quite low (26.84\%). Chinese, the second most researched language, has 52\% \textit{Class I} papers and only 26\% \textit{Class III} papers. On the other hand, for not-so-resource-rich languages like Deutsch, the percentages of \textit{Class III} papers for Deutsch are quite high (44.74\%) compared to English and Chinese.
\\

\begin{figure}[t]
\centering
\includegraphics[width=12cm]{figure7.pdf}
\caption{Replication Material Availability by Language}\label{fig:7}
\end{figure}

\noindent
The detailed statistics are shared in our GitHub repository. Overall, researchers are getting more interested in making their resources (code, data, documentation, etc.) publicly available over time, irrespective of the underlying language in question.

\section*{Legal NLP Citation Analysis}
As highlighted in herein, the field of Legal NLP is growing in many interesting ways. To help round out our analysis and understand which papers are commonly referenced by scholars, we collected basic citation data.  Citations can provide a useful perspective to help identify key papers within the field.    While citations are a noisy measure of `quality,' they do reflect a crowd-sourced measure of prominence within the scientific community.
\\
\\
Leveraging Google Scholar as a source for citation information, we automated the collection of citation data while qualitatively reviewing the outputs of this automation for quality control purposes.  Citations counts reflect the current status as of early January 2023.  Across all of the papers, citation counts widely varied but something akin the classic bibliometric skew\cite{brzezinski2015power} towards the top papers is present within the field of Legal NLP.
\\
\\
In reviewing the raw citations patterns, it was clear that the age of a publication was a contributing factor to citation count.  This is, of course, not surprising as the longer a paper exists, the more opportunities a paper has to be cited.  To help correct for this issue, we normalized citation counts.  Although more sophisticated normalization methods are certainly possible,\cite{owlia2011normalized} we conducted linear normalization of the citations counts for each paper in our corpus.  All citation data will be made available in our online repository.
\\
\\
Table \ref{tab:most_cited} offers a normalized list of the most cited papers in field authored in the past decade.  Even a cursory review reflects the wide variety of substantive topics and jurisdictions.  Collectively, these papers represent a good initial starting point for those interested in learning more about the emerging field of Legal NLP.

\begin{table*}
\centering
    \resizebox{\textwidth}{!}{
    \begin{tabular}{l|p{10cm}|p{7cm}|l}
    \toprule
    \bf Rank & \bf Title & \bf Authors & \bf Year\\
    \midrule
1	& Predicting judicial decisions of the European Court of Human Rights: a Natural Language Processing perspective &	Aletras,	Tsarapatsanis,	Preoţiuc-Pietro, Lampos &	2016\\
\midrule
2	& LEGAL-BERT: The Muppets straight out of Law School &	Chalkidis, Fergadiotis	Malakasiotis, Aletras, Androutsopoulos &	2020 \\
\midrule
3	& Using machine learning to predict decisions of the European Court of Human Rights &	Medvedeva,	Vols,	Wieling	& 2020\\
\midrule
4	& How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence &	Zhong, 	Xiao	Tu, Zhang, Liu, Sun &	2020\\
\midrule
5	& Legal Judgment Prediction via Topological Learning & Zhong, Haoxi, Guo, Tu, Xiao, Liu, Su &	2019 \\
\midrule
6	& LexGLUE: A Benchmark Dataset for Legal Language Understanding in English &	Chalkidis, Jana,	Hartung, Bommarito, Androutsopoulos, Katz, Aletras &	2022\\
\midrule
7	& Neural Legal Judgment Prediction in English & Chalkidis, Androutsopoulos, Aletras &	2019 \\
\midrule
8	& Learning to Predict Charges for Criminal Cases with Legal Basis &	Luo, Feng, Xu, Zhang, Zhao	& 2017\\
\midrule
9	& Few-Shot Charge Prediction with Discriminative Legal Attributes & 	Hu	Li	Tu, Liu, Sun	& 2018 \\
\midrule
10	& Natural language processing to identify the creation and impact of new technologies in patent text: Code, data, and new measure &	Arts, Hou, Gomez &	2021  \\
\midrule
11	& TechNet: Technology semantic network based on patent data &	Sarica, Luo, Wood &	2020 \\
\midrule
12	& Text summarization from legal documents: a survey & Kanapala, Pal, Pamula &	2019\\
\midrule
13	& Patent claim generation by fine-tuning OpenAI GPT-2 & Lee, Hsiang & 2020\\
\midrule
14	& CLAUDETTE: an automated detector of potentially unfair clauses in online terms of service & 	Lippi, Palka, Contissa, Lagioia, Micklitz, Sartor, Torroni &	2019 \\
\midrule
15	& A comparative study of automated legal text classification using random forests and deep learning &	Chen, Wu, Chen, Lu, Ding &	2022\\
\midrule
16	& Exploring the Use of Text Classification in the Legal Domain & Sulea, Zampieri, Malmasi, Vela, Dinu, van Genabith	& 2017 \\
\midrule
17	& When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset of 53,000+ Legal Holdings & Zheng, Guha, Anderson, Henderson, Ho	& 2021\\
\midrule
18 & Deep learning in law: early adaptation and legal word embeddings trained on large corpora & Chalkidis, Kampas	& 2018\\
\midrule
19	& Legal Judgment Prediction via Multi-Perspective Bi-Feedback Network &	Yang, Jia, Zhou, Luo &	2019 \\
\midrule
20	& Distinguish Confusing Law Articles for Legal Judgment Prediction & Xu, Wang, Chen, Pan, Wang, Zhao &	2020 \\
\midrule
21	& CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review & Hendrycks, Burns, Chen, Ball	& 2021\\
\midrule
22	& BERT-PLI: Modeling Paragraph-Level Interactions for Legal Case Retrieval & Shao, Mao, Liu, Ma, Satoh, Zhang, Ma & 2020\\
\midrule
23	& Lawformer: A pre-trained language model for chinese legal long documents & Xiao, Hu, Liu, Tu, Sun & 2021\\
\midrule
24	& EC-QA: a legal-domain question answering dataset & Zhong, Xiao, Tu, Zhang, Liu, Sun	& 2020\\
\midrule
25	& Extracting Contract Elements & Chalkidis, Androutsopoulos, Michos & 2017\\
\midrule
26	& Automated requirements identification from construction contract documents using natural language processing & 	Hassan, 	Le		& 2020\\
\midrule
27	& Fine-Grained Named Entity Recognition in Legal Documents & Leitner, Rehm, 	Moreno-Schneider &	2019\\
\midrule
28	& Iteratively Questioning and Answering for Interpretable Legal Judgment Prediction &	Zhong, Wang, Tu, Zhang, Liu, Sun & 2020\\
\midrule
29	& Legal Area Classification: A Comparative Study of Text Classifiers on Singapore Supreme Court Judgments & Soh, Lim, Chai	& 2019\\
\midrule
30	& VICTOR: a dataset for Brazilian legal documents classification & Araujo,	de Campos, Braz, DaSilva &	2020\\
\bottomrule
\end{tabular}
}
\caption{Most Cited Papers in Legal NLP Published Since 2013 (Normalized)}
\label{tab:most_cited}
\end{table*}

\section*{An Interactive Living Survey}
A survey spanning a decade of global research is an enormous task,
even in an emerging field such as Legal NLP.  The sheer amount of information creates a significant risk of overburdening the community,
resulting in scientific findings being overlooked or getting lost.
Even at the beginning of the period under review,  the growth rate of publications in our field (see infra \ref{fig:1}) made it difficult to identify all relevant publications.  As the number of new publications over time grew and methodical and linguistic diversity increased, the task has become constantly more challenging. Similar developments in science in general \cite{bornmann2015growth} have inspired a series of innovative attempts to solve the problem of scientific knowledge discovery and education \cite{thoppilan2022lamda} by using of large language models to support both research \cite{beltagy2019scibert} and writing of scientific publications \cite{taylor2022galactica} albeit with limited success and apparent upper bounds despite large numbers of parameters at present.\cite{hong2022scholarbert} Given current technical limitations and employing an established design mechanism in machine learning, \cite{wu2022survey,wang2022human}  we suggest a hybrid, human-in-the-loop approach to information management, combining state-of-the-art natural language processing tools to find and curate publications for review with subject matter expert human control.  This idea of a \emph{living} survey has been successfully implemented in other domains of machine learning research such as deep neural networks \cite{wang2019visual} and natural language processing research such as explainable artificial intelligence. \cite{qian2021xnlp}
\\
\\
While we are confident that our extensive review of the existing literature has yielded a comprehensive collection of papers,
the fast publication pace in the field as well as the linguistic diversity of legal research makes it probable that we may have missed relevant, individual contributions.
For this survey to be a as helpful for the community as possible we have therefore built web infrastructure\footnote{Available at URL: \%\%\% (Coming Soon)} to accept additional contributions from the public and continuously update our results accordingly.
Researchers can provide a link to or upload their contribution, fill in a small number of fields with meta-data so that the task of maintaining the collection can be distributed across the community building on successful models of operation from the open source movement.
\\
\\
This digital infrastructure provides us with an opportunity to not only create a living but also an \emph{interactive} survey. Since publications in the field of computational legal studies and natural legal language processing should be more than mere papers and ideally contain both code and data for reproduction, \cite{coupette2022sharing} we have a provided an easy-to-navigate graphical interface to explore the collections of papers contained in this review. Users can search publications and filter the collection according to their needs and based on the taxonomy laid out in Table \ref{tab}. In addition, they can select individual and sets of publications to display temporal dynamics in methods, topics and languages.

\section*{Conclusion and Future Perspectives}

As highlighted above, the field of Legal NLP is growing in volume, diversity of languages and sophistication of methods.
As a result, research is conducted on an increasing variety of tasks as scientists push the limits of technical feasibility and demonstrate successful resolutions of increasingly difficult real-world challenges.\cite{bommarito2022gpt}
Over our period of observation, an ever-growing amount of legal data and computational resources have become publicly available, certainly contributing to major advances in the field. However, the effects of training data, model architectures, and modeling techniques compared to the continuous increase in scale of general models requires extensive further research.
\\
\\
Large language models have recently fully entered the public's perception, resulting in viable commercial interest from players historically disinterested in Legal NLP research, such as publishing houses, law firms and courts.
They often possess vast collections of data and might be increasingly willing to share them with researchers.
This, in turn, is likely to lead to an uptick in research using real-world, commercial, or administrative datasets.
While commercial players will drive the interest in Legal NLP research their very own challenge will lie in the connection of the resulting, ever increasing technical capabilities into products.
Among the broader world of legal technology, it is therefore language-centric technologies that are likely to play an ever-increasing role.
Public players will likely focus on deployment in the context of digital justice, leveraging models of the neural era to reduce case backlogs, improve access, and develop the ability to deliver justice at scale.
\\
\\
Our analysis has revealed that legal text generation has played a rather limited role over the last decade.
With generative models capturing much of the attention recently, Legal NLP research will likely reflect this interest.
As we demonstrated the usefulness of NLP methods for literature surveys themselves, one potential avenue for further research is the role of generative models for science itself and the division of labor between human researchers and large language models in our very own domain.
\\
\\
Concerning the engineering part of our survey, future work consists of two tasks: updating the \emph{living }survey with state-of-the-art publications and building and releasing new \emph{interactive} features for exploration.
While we hope that the community can assist in the former we plan to focus on the latter,
starting with a functionality to extract and visualize the reference graph of our collection and corresponding network metrics.
\\
\\
All of the above developments finally point to a growing awareness and better understanding of computational legal studies among traditional legal as well as other empirically-minded scholars.  This might well result in an increased intra-disciplinary collaboration and a greater openness toward quantitative methods, thereby fostering the relevance of Legal NLP in the academic community.

\section*{Data Availability}
All data collected in this survey will be made available on GitHub upon paper publication.

\section*{Acknowledgements}
This work was partially funded by the German Federal Ministry of Education and Research (BMBF) through the \href{https://www.hilano.de}{HILANO} project (funding code 01IS18085).

\vspace{.1pt}

redpajama_set_name: RedPajamaArXiv


text:
E-matic's new MP3 player comes in iPod like shades.. but includes a camera!
We're used to MP3 players that come in multiple colours by now (thanks Apple) have an interesting interface (the Spinn comes to mind) and double up as PMP's (where do I even start with this one?) but one that also integrates a CAMERA and video camera is a new idea. That's what the E-matic EM324CAMP promises to do, as within its sleek shell it hold a 2 megapixel camera and video option that can record up to 70 hours of video, in 7 different modes. You can choose from Sepia, Black/White, Solaris and more for those 'real' film like effects and then you can view your masterpiece on the 2.4 inch screen.
There's also a built in speaker if you want to share your efforts with your friends as well as a dedicated 3.5mm jack, so you can use whatever 'phones you wish with it. It has 4GB of internal memory and is expandable via the MicroSD card slot. File-wise it supports a selection of formats, from MP3, WMA, AVI, MP4, Jpeg, BMP's and more. It also says it's e-book compatible, but as there's no e-ink reader I'd probably avoid this option.
For such a teeny price I have to admit it sounds very tempting; shame it's only available in the US though.
$60 from E-Matic
See here for more MP3 players
Nov 3, 2008 Zara Rabinowicz
Day One: Coming off the social network crack
Carphone Warehouse stocks Toshiba's NB100 with a competitive price plan
i cant get this thing to work with my videos, its rediculous, ive gone throu the specifications it says to use but still nothing….i like the mp3 player but if i cant get my videos to work its going right back to the store! ive even downloaded the necissary converters in order to convert my videos to those specifications and still doesnt work! i need an answer soon by then end of this month or im returning it! email me at sukyakakitsuka@yahoo.com
How do you delete things from these things? I can get it to take pics but don't know what to do with them after I take them.
I was wondering how to change the brightness and contrast on the pictures… also…how do you load the music… I'm about to get my 14 yr old nephew to do all this for me.. I am very computer literate but I can't figure this thing out…please any help would be appreciated.
IT HAS PRETTY COLORS BUT I CANT GET THE MUSIC TO SHOW UP ON THE PLAYER BUT IT SAYS THAT THE MUSIC IS ON THE PLAYER

redpajama_set_name: RedPajamaCommonCrawl


text:
Дмитро Федорович Лупішко — український астроном, засновник харківської астероїдної наукової школи, керівник відділу фізики астероїдів і комет Харківської обсерваторії (1989—2012), лауреат Державної премії України в галузі науки і техніки (2010).

Життєпис
Народився 15 лютого 1942 року в селі Кошари Сумської області. 1969 року закінчив фізичний факультет Харківського державного університету та поступив до аспірантури під керівництвом академіка
М. П. Барабашова, але не встиг захиститись під його керівництвом — Барабашов помер 1971 року. 1975 року Дмитро Лупішко захистив кандидатську дисертацію «Абсолютна інтегральна та поверхнева фотометрія Марса в протистоянні 1971 р.», а 1999 року — докторську дисертацію «Фотометрія і поляриметрія астероїдів: результати спостережень та аналіз даних». З 1993 по 2006 рік завідував відділом фізики астероїдів і комет НДІ астрономії.

Почавши наукову кар'єру з досліджень Марса, з часом почав вивчати астероїди, ставши першим науковцем СРСР, який систематично досліджував саме астероїди. 1984 року створив і очолив робочу групу «Астероїди» при Астрораді Академії наук СРСР. В наступні роки Дмитро Лупішко організував та провів чотири засідання цієї робочої групи, а також дві Всесоюзні семінар-наради. Пізніше він організував проведення в Харківській обсерваторії Міжнародних робочих груп «Поляриметрія комет і астероїдів» (1997) і «Фотометрія і поляриметрія астероїдів» (2003).

В фізиці астероїдів Дмитро Лупішко дослідив закони розсіяння світла астероїдними поверхнями, виявив кілька нових поляризаційних ефектів. Показав, що поверхні астероїдів М-типу не є суто металевими, а містять значну частину силікатів. Поліпшив значення діаметрів і альбедо астероїдів, отриманих інфрачервоним космічним телескопом IRAS. Створив поляриметричний банк астероїдних даних, що зараз став частиною бази даних .

На день 80-річчя Лупішка в НДІ астрономії ХНУ організовано робочу групу «Фотометрія і поляриметрія астероїдів: минуле і майбутнє».

Відзнаки
 Державна премія України в галузі науки і техніки (2010).
 Премія НАНУ ім. акад. М. П. Барабашова (1988)
 На честь науковця названо астероїд 3210 Лупішко.

Примітки

Посилання
 До ювілею видатного українського астронома: професор Дмитро Лупішко про дослідження астероїдів

 Dmitrij F. Lupishko , Institute of Astronomy of Kharkiv National University

 Лупішко Дмитро Федорович, Енциклопедія сучасної України
 Lupishko, Dimitrij F., профіль в Scopus

Українські астрономи
Випускники Харківського університету
Науковці Харківського університету
Люди, на честь яких названо астероїд
Науковці Харківської астрономічної обсерваторії

redpajama_set_name: RedPajamaWikipedia


With John Deere you are literally spoilt for choice when it comes to baler options. These balers are manufactured to withstand anything you can throw at them and produce consistent results day in, day out.
John Deere balers show outstanding reliability in all conditions. Durable welded frames will take all the abuse your farm or contracting business dishes out and the unique, reliable design ensures a smooth uniform crop flow producing superior forage quality. From small livestock farms to high volume contractor fleets, you'll find the perfect match with a John Deere baler.
At Smithfield Tractors we focus on the 440M and 440R round baler models, but don't forget, with John Deere there is a baler to suit every budget and every farming operation. Each with a choice of different pick up widths, capacities and monitoring and control systems. All built to the same uncompromising John Deere standards.
For more information click here to visit the John Deere Website and for expert advice speak to one of our team here at Smithfield Tractors.
High capacity. Heavy duty design.


This is simple to use music controller.
Sound packs are included and configured.
Just use your creativity and get fun!
Trance edition of your favorite music app is out!
Now it's universal app - for iPhone and iPad!
Enjoy creating your music by touching pads.
Amaze your friends - become a DJ on your party!


text:
First Look Review - HILLBILLY HORROR SHOW: Vol. 4
Latest volume of the comic horror anthology series.
Review by Andy Comer
Directed by: Sharif Salama
Starring: Bo Keister, Scott Geiter, Rachel Faulkner
"If this show does continue I hope they can at the very least put more creativity into their shorts to help cover up the pathetic hillbilly banter between the hosts."
Having not seen any of the previous instalments of this show, I had no idea what I was in for. I assumed since it had made its way to a fourth installment that it had some sort of redeemable concept to warrant the production of multiple outings, be it hilarious hillbillies chewing tobacco, beer, and racing. Or perhaps there are genuinely enjoyable horror stories being told. Sadly, neither of these things are happening here.
The show is set up kind of like those old programs where hosts would tell anecdotes to each other while getting ready to put on a film. Then, as breaks in the film take place, we would catch up with the hosts, who would discuss what has been happening either in the film being presented or with shenanigans on the set. Here, it is a bit of both, and instead of relying on being funny or entertaining, the hosts spend their screen time making jokes that fall flat, but I doubt that they care because they have an extremely attractive young lady moving around the screen in different outfits, only stopping in the perfect positions for the camera to capture her, uh-hum curves. She is one of the few redeemers here because it is abundantly clear what she is there for and she does it exceptionally well.
When it comes to the actual shorts the hosts are presenting, I am reminded of the segmented horror features that have become popular the past few years; most notably The ABCs of Death series and the VHS series. The first of the shorts is called Strange Thing, in which a couple discovers a door in their home, and once they go inside they discover some "Thing" that ends up attacking them. This one was decent, though I would have liked them to do more with the creature. The short ends up being too shallow to be memorable. The second, and best, of the shorts is a horror comedy called 'Til Death where four friends off each other's significant others with very odd results. For what it was it is definitely unique and I could see something like it working at feature length.
From there it just goes downhill. In the next short, Foodie, an annoying restaurant owner goes to a dinner with a shocking menu, but the story and interactions between characters is extremely dull. The last of the traditional shorts is The Hourglass Figure. I have no clue why they added "figure" to the title, because it sounds clunky, but clunky is normal for these guys because the story was a pretty uneven one about a mother who uses a magical hourglass to find more time in her busy schedule. I mentioned traditional before that last one because there is a "short" short called Collision Course and while it was shorter than the others it was pretty effective with its limited time, showing what the last seven days on earth would be like before it collided with Jupiter.
Ultimately, with its hour and a half run time I would say that there is about 20 minutes of quality in total. Basically, with the lows of the show dipping far lower in comparison than the heights of the good aspects of the runtime I cannot recommend this one. If this show does continue I hope they can at the very least put more creativity into their shorts to help cover up the pathetic hillbilly banter between the hosts.
Labels: By Andy Comer, First Look, horror, New Releases


text:
BreakingMuscle UK
CrossFit, Sponsorships, and the Ugly Truth About Gender Inequality
Patrick McCarty
CrossFit, Masters Athletes
CrossFit, Mature Athletes
The CrossFit Journal recently published a piece about female athletes entitled "Frailty, Thy Name Is Woman?" The article challenged the idea that women are the weaker sex. In the article, Elisabeth Akinwale said:
I share sponsors with some women who have never done anything athletically. They train ... and they post a lot of videos of their cleavage and stuff, and the camera angle going up toward their butt, but they are not successful athletes...
[S]o, while I understand it, it's sort of frustrating. … Are there any men (in CrossFit) who are sponsored who are just basically hot? I can't think of any.
It may read as if Akinwale is taking potshots at women, but her criticisms are directed more at the gender inequality of sponsorship and less at other females. The general sentiment, especially clear in the last sentence, is that men don't have to be hot to get sponsored, they just need to be tremendous athletes. Women, on the other hand, can and do get sponsorships based more on their looks than athletic ability.
That "sex sells" is nothing new. What is concerning is that so many women in the CrossFit world have seemingly bought into that idea, opting for exploitation of their sexuality over athletic performance in order to garner likes, followers, and hits. And to Akinwale's point, those who don't chose this path sometimes pay the economic price.
A Real-Life Example
For example, in 2011 Reebok produced its first shoe commercial featuring CrossFit athletes. The winners of the 2010 CrossFit Games were Graham Holmberg and Kris Clever. When the commercial was produced, Reebok featured Holmberg and…Becca Voigt.
Why was 2010 champion Kris Clever passed up for a national commercial? I'll tell you why. Because sex sells, and Reebok felt that Voigt, as a tall and attractive female, would move more units. I wonder sometimes how that impacted Clever. Perennial Games competitor, 2010 champion, GRID athlete for the LA Reign. A clever person who, from my best estimation, has few sponsors for someone as athletically talented as she is.
Kris Clever at the 2012 CrossFit Games.
A Complex Issue
Over the next three weeks, I'll take a look at the following issues one at a time:
Gender inequality in sponsorships and opportunities – Akinwale's critique. (Week 1)
The sex-sells trap in CrossFit. (Week 2)
The larger, more complicated issue of sexualisation of women in sports and its pitfalls. (Week 3)
Akinwale's point was that men are awarded sponsorship contracts on the basis of athletic talent, not looks. The criteria for sponsored female athletes is more nebulous, and includes athletic ability, looks, and sex appeal. Akinwale is 100 percent correct, and the Holmberg/Clever commercial is clear evidence. By all accounts, Holmberg and Clever's "athletic achievement" was equal in 2010, yet Holmberg was chosen for the national spot and Clever was not. It all boils down to this question: how many units can be moved?
"Female athletes have to play two games: one, their chosen sport, and two, the game of attraction."
Akinwale goes on to comment in the Facebook thread: "Female athletes' economic outcomes are more impacted by physical appearance and mainstream beauty standards than men's. Period." Again, this is indisputable. Let's restate her comment with men out of the equation: "Female athletes' economic outcomes are impacted by physical appearance and mainstream beauty standards." Female athletes have to play two games: one, their chosen sport, and two, the game of attraction. Attractiveness, sexuality, and the willingness to use those qualities as marketing all figure into the equation of economic reward for female athletes.
It's Not Just CrossFit
Don't believe me? Check out what swimmer Ana Rodrigues said in her 2013 article, "Female Athletes Still Face Inequality":
In some ways, good looks have led many female athletes to come into the spotlight, seemingly making appearances more important than skills. The women that are willing to show a little skin are the ones that are most recognisable and even though they may have the athletic skills, they only gain popularity after their exposure (literally).
Or consider Olympic snowboarder Hannah Teter. She enjoyed modest fame on the basis of her gold medal, but that recognition was nothing like the popularity she received after posing for Sport's Illustrated's swimsuit issue. Look up racing driver Danica Patrick or tennis player Anna Kournikova. Or read up on Olympic gold medal hurdler Sally Pearson, who has 32.6k Instagram followers. Christmas Abbott, by all accounts a marginal athlete in the CrossFit realm, has 290k followers. The list goes on and on – average athletes who became the sport's most popular female athletes on the basis of their appearance.
I will give you one guess as to why, much of which will be explored in Part II. There is even a documentary, Branded, about this very thing:
Women: It's Up to You
Let's revisit the subheading of the original article in which Akinwale was quoted:
In CrossFit gyms around the world, women deconstruct the longstanding myth of "the weaker sex" and continue the march toward true equality.
But women in the sport are not marching in step. If a CrossFit athlete knowingly uses sex over athleticism to brand and market themselves, she does not continue the "march toward true equality." As the article states:
Cultural changes are needed as well. True equality has not yet arrived when the conversation surrounding the best female tennis players in the world is about their appearance.
Where will those changes come from? The sponsors? No. They only care about revenue. The sport itself? Laughable. CrossFit, Inc. thrives on its misogynistic portrayal of women. No, the change will come from women. The "march toward true equality" begins with the marcher. At least Elisabeth Akinwale can honestly say she's on the march.
What It Really Takes to Get to the CrossFit Games
Speaking Out: Charlie Hebdo and CrossFit
Camille Leblanc-Bazinet: Strong, Real, and Less-Than-Perfect
New on Breaking Muscle UK Today
Photo courtesy of CrossFit, Inc.
See more about: gender, female athletes, women, crossfit, sponsorship, functional fitness
Handstand Strength
Rowing for Weight Loss
Ignite: Fat Loss
28 Day Clean Eating Challenge
The Complete Pregnancy
Outcry Over Newspaper's Coverage of Tia-Clair Toomey
3 Training Truths You Know But Aren't Doing
Don't Judge the Judging: Do Something About It
4 Principles to Master the Rowing Machine
There Are No Tricks in Fitness
Dave Matthews, the Elliptical Machine, and the Haters
How to Train for the Zombie Apocalypse
The Unwritten Rules of the CrossFit Open
3 Reasons You Don't Squat More (And What to Do About It)
Gwen Sisto: A True Story of When Olympic Weightlifting Meets Juggling
Mommy Workout: First Trimester - Week 4, Day 2
Women's Workout: Cycle 5, Week 3, Day 2
Mommy Workout: First Trimester - Week 10, Day 2
Athlete Journal: Chris Duffin, Entry 29 - Time to Deload
Athlete Journal: Charles Staley, Entry 23 - Working On My Squat


Operators can be applied to constants, cell references, or functions.
The results of any of these comparisons will be a logical value – either True or False.
The Intersection of two ranges is a reference to all the cells that are common between the two ranges.
The order in which a calculation is performed may affect the result. As an example, the calculation 6+4*2 could be interpreted in two different ways. If the addition is performed first, this would give 10*2, which equals 20. However, if the multiplication is performed first, the calculation becomes 6+8, which equals 14.
Operator precedence is a mathematical concept used by all programming languages and applications, such as spreadsheet programs that include computation.
When the formula has several operators with the same precedence, multiplication and division for example, Excel evaluates the operators from left to right.
You use parentheses to change the order of evaluation, since the expressions within parentheses are evaluated first. If there are parentheses within parentheses, Excel evaluates the expression in the innermost pair of parentheses first, then works outwards.
Operator precedence in Excel is not always the same as mathematical precedence. For example, take the expression -32. This is entered as -3^2 which Excel evaluates as (-3)^2 which is +9. However, mathematically you'd expect -(3^2) which is -9.
For the complete guide to Excel 2016, all in the trusted In Easy Steps style, click here. Whether you're upgrading from a previous version or new to the spreadsheet concept, Excel 2016 in easy steps will help you learn the key features constructively so you'll be a pro in no time!


BREAKING NEWS: Is Your Neighborhood Favorite Open or Closed Tonight?
So we've made our peace with the fact that we're not going to be able to make it over to Corduroy for our planned birthday dinner tonight. Even though Chef Tom Power and his intrepid crew are open for business, we're taking the mayor's (and everyone else's) advice and staying off the roads.
But we've still got to get out there and celebrate somewhere, right? And so do you.
If you're planning to keep it local and check out one of your neighborhood establishments, you should probably check first and see if they're open before you bundle up and head out.
We're here to help. We'll be updating this post with a list of restaurants https://capitalspice.wordpress.com/wp-admin/post.php?action=edit&post=4170&message=1that are definitely open or closed tonight as we hear of them. Know of others? Let us know in the comments section.
A few months ago, we wrote about one of our consistent favorites in Penn Quarter, PS 7's. We praised Chef Peter Smith's approach to his menu and his commitment to using the best ingredients available to make inventive dishes with satisfying, complex flavors. We mentioned Gina Chersevani's cocktails in passing, but we largely focused on the meal.
For a lot of guests at PS 7's, that's the way they experience the menu. But it doesn't have to be that way. A few of our fellow bloggers (as well as the not-long-for-DC Daily Candy) have written about a way to have your cake and eat it too, in a manner of speaking. If you know to order it, you can actually enjoy a seven-course tasting menu that comes paired with seven of Gina's amazing cocktails in the lounge.
Don't expect to see a menu (they abandoned the mysterious wax-sealed menus that the guys at the Scofflaw's Den experienced), but do come prepared to eat and drink well.
We know, we know…we promised to announce the winner of our Top Chef DC Elimination Challenge this morning. We were blown away by your response: more than 80 suggestions that touched on almost every aspect of Washington life. Our word is good – we'll announce the winner shortly.
But since we've got your attention, we wanted to take a minute to tell you about a terrific way for you to help out one of our favorite local charities. BGR has teamed up with the DC Central Kitchen to put on a unique promotion for the month of December. Throughout the month, BGR will donate one hamburger's worth of their prime ground beef for every burger purchased in all three of their locations (Bethesda, Old Town and Dupont Circle).
DC Central Kitchen has been "Feeding the Soul of the City" for 20 years now, offering hot meals to the homeless and quite a bit more. Their Fresh Start Catering and Capital Carts programs provide training, education and job opportunities in the culinary industry for individuals trying to turn their lives around. And their relationships with local restaurants and farmers' markets allow them to make use of over a ton of donated food every day.
What does this cost you? Absolutely nothing. Buy the same burger you normally would, and BGR will make the contribution. You don't need to mention a secret word or turn in some kind of coupon or even know you're doing it…BGR will give for every burger sold, period.
Sure, the DC dining scene is positively overrun with burger joints. Chances are there's at least one or two other options between you and your nearest BGR. And we're not about to tell you which establishment should be your favorite (that's an intensely personal decision that every person needs to make for himself or herself). But how many of BGR's competitors are stepping up with this kind of charitable giving?
If you've never tried Mark Bucher's badass burgers, this is a perfect excuse to check them out for yourself. We experienced BGR for the first time last month, and it's a safe bet we'll be back to check out some of their other burgers soon enough. And – we can't stress this enough – try it with the Orange Standard. Those sweet potato fries were GREAT.


text:
[Licensee::Projects::FSProject,
 Licensee::Projects::GitProject].each do |project_type|
  RSpec.describe project_type do
    let(:mit) { Licensee::License.find('mit') }
    let(:other) { Licensee::License.find('other') }
    let(:fixture) { 'mit' }
    let(:path) { fixture_path(fixture) }
    subject { described_class.new(path) }

    if described_class == Licensee::Projects::GitProject
      before do
        Dir.chdir path do
          `git init`
          `git config --local commit.gpgsign false`
          `git add .`
          `git commit -m 'initial commit'`
        end
      end

      after do
        subject.close
        FileUtils.rm_rf File.expand_path '.git', path
      end
    end

    if described_class == Licensee::Projects::GitProject
      context 'when initialized with a repo' do
        let(:repo) { Rugged::Repository.new(path) }
        subject { described_class.new(repo) }

        it 'returns the repository' do
          expect(subject.repository).to be_a(Rugged::Repository)
        end
      end

      context 'when initialized with a revision' do
        let(:revision) { subject.repository.last_commit.oid }
        before do
          subject.instance_variable_set('@revision', revision)
        end

        it 'returns the commit' do
          expect(subject.send(:commit).oid).to eql(revision)
        end
      end
    end

    it 'returns the license' do
      expect(subject.license).to be_a(Licensee::License)
      expect(subject.license).to eql(mit)
    end

    it 'returns the matched file' do
      expect(subject.matched_file).to be_a(Licensee::ProjectFiles::LicenseFile)
      expect(subject.matched_file.filename).to eql('LICENSE.txt')
    end

    it 'returns the license file' do
      expect(subject.license_file).to be_a(Licensee::ProjectFiles::LicenseFile)
      expect(subject.license_file.filename).to eql('LICENSE.txt')
    end

    it "doesn't return the readme" do
      expect(subject.readme_file).to be_nil
    end

    it "doesn't return the package file" do
      expect(subject.package_file).to be_nil
    end

    context 'reading files' do
      let(:files) { subject.send(:files) }

      it 'returns the file list' do
        expect(files.count).to eql(2)
        license = files.find { |f| f[:name] == 'LICENSE.txt' }
        expect(license).to_not be_nil

        if described_class == Licensee::Projects::GitProject
          expect(files.first).to have_key(:oid)
        end
      end

      it "returns a file's content" do
        content = subject.send(:load_file, files.first)
        expect(content).to match('Permission is hereby granted')
      end

      if described_class == Licensee::Projects::FSProject
        context 'with search root argument' do
          let(:fixture) { 'license-in-parent-folder/license-folder/package' }
          let(:path) { fixture_path(fixture) }
          let(:license_folder) { 'license-in-parent-folder/license-folder' }
          let(:search_root) { fixture_path(license_folder) }
          let(:subject) { described_class.new(path, search_root: search_root) }
          let(:files) { subject.send(:files) }

          it 'looks for licenses in parent directories up to the search root' do
            # should not include the license in 'license-in-parent-folder' dir
            expect(files.count).to eql(1)
            expect(files.first[:name]).to eql('LICENSE.txt')
          end
        end

        context 'without search root argument' do
          let(:fixture) { 'license-in-parent-folder/license-folder/package' }

          it 'looks for licenses in current directory only' do
            expect(files.count).to eql(0)
          end
        end
      end
    end

    context 'encoding correctness' do
      let(:fixture) { 'copyright-encoding' }

      it "returns a file's content" do
        expect(subject.license_file.content).to match(
          'Copyright © 2013–2016 by Peder Ås, 王二麻子, and Seán Ó Rudaí'
        )
      end
    end

    context 'readme detection' do
      let(:fixture) { 'readme' }
      subject { described_class.new(path, detect_readme: true) }

      it 'returns the readme' do
        expect(subject.readme_file).to be_a(Licensee::ProjectFiles::ReadmeFile)
        expect(subject.readme_file.filename).to eql('README.md')
      end

      it 'returns the license' do
        expect(subject.license).to be_a(Licensee::License)
        expect(subject.license).to eql(mit)
      end
    end

    context 'package manager detection' do
      let(:fixture) { 'gemspec' }

      # Using a `.gemspec` extension in the fixture breaks `gem release`
      before do
        FileUtils.cp("#{path}/project._gemspec", "#{path}/project.gemspec")
        if described_class == Licensee::Projects::GitProject
          Dir.chdir path do
            `git add project.gemspec`
            `git commit -m 'add real gemspec'`
          end
        end
      end

      after do
        FileUtils.rm("#{path}/project.gemspec")
      end

      subject { described_class.new(path, detect_packages: true) }

      it 'returns the package file' do
        expected = Licensee::ProjectFiles::PackageManagerFile
        expect(subject.package_file).to be_a(expected)
        expect(subject.package_file.filename).to eql('project.gemspec')
      end

      it 'returns the license' do
        expect(subject.license).to be_a(Licensee::License)
        expect(subject.license).to eql(mit)
      end
    end

    context 'multiple licenses' do
      let(:fixture) { 'multiple-license-files' }

      it 'returns other for license' do
        expect(subject.license).to eql(other)
      end

      it 'returns nil for matched_file' do
        expect(subject.matched_file).to be_nil
      end

      it 'returns nil for license_file' do
        expect(subject.license_file).to be_nil
      end

      it 'returns both licenses' do
        expect(subject.licenses.count).to eql(2)
        expect(subject.licenses.first).to eql(Licensee::License.find('mpl-2.0'))
        expect(subject.licenses.last).to eql(mit)
      end

      it 'returns both matched_files' do
        expect(subject.matched_files.count).to eql(2)
        expect(subject.matched_files.first.filename).to eql('LICENSE')
        expect(subject.matched_files.last.filename).to eql('LICENSE.txt')
      end

      it 'returns both license_files' do
        expect(subject.license_files.count).to eql(2)
        expect(subject.license_files.first.filename).to eql('LICENSE')
        expect(subject.license_files.last.filename).to eql('LICENSE.txt')
      end
    end

    context 'lgpl' do
      let(:gpl) { Licensee::License.find('gpl-3.0') }
      let(:lgpl) { Licensee::License.find('lgpl-3.0') }
      let(:fixture) { 'lgpl' }

      it 'license returns lgpl' do
        expect(subject.license).to eql(lgpl)
      end

      it 'matched_file returns copying.lesser' do
        expect(subject.matched_file).to_not be_nil
        expect(subject.matched_file.filename).to eql('COPYING.lesser')
      end

      it 'license_file returns copying.lesser' do
        expect(subject.license_file).to_not be_nil
        expect(subject.license_file.filename).to eql('COPYING.lesser')
      end

      it 'returns both licenses' do
        expect(subject.licenses.count).to eql(2)
        expect(subject.licenses.first).to eql(lgpl)
        expect(subject.licenses.last).to eql(gpl)
      end

      it 'returns both matched_files' do
        expect(subject.matched_files.count).to eql(2)
        expect(subject.matched_files.first.filename).to eql('COPYING.lesser')
        expect(subject.matched_files.last.filename).to eql('LICENSE')
      end

      it 'returns both license_files' do
        expect(subject.license_files.count).to eql(2)
        expect(subject.license_files.first.filename).to eql('COPYING.lesser')
        expect(subject.license_files.last.filename).to eql('LICENSE')
      end
    end

    context 'with a copyright file' do
      let(:fixture) { 'mit-with-copyright' }

      it 'returns MIT' do
        expect(subject.license).to eql(mit)
      end
    end
  end
end

redpajama_set_name: RedPajamaGithub