Specifications TableSubject*Information Systems*Specific subject area*Data mining, Digital History, Labeled Dataset for Machine Learning*Type of dataTable.How data were acquiredPast causalities described by authors by referencing past causalities in some Japanese Textbooks for World History\
Past events are crawled from WikipediaData formatFiltered\
Raw data of [Fig. 1](#fig1){ref-type="fig"}, [Fig. 2](#fig2){ref-type="fig"}, [Fig. 3](#fig3){ref-type="fig"}, [Fig. 4](#fig4){ref-type="fig"}, [Fig. 5](#fig5){ref-type="fig"}, [Fig. 6](#fig6){ref-type="fig"} and [Table 4](#tbl4){ref-type="table"} are stored in:\
causality_regional_distribution.tsv ([Fig. 1](#fig1){ref-type="fig"}),\
causality_temporal_distribution.tsv ([Fig. 2](#fig2){ref-type="fig"}), and Statistics.tsv ([Fig. 3](#fig3){ref-type="fig"}, [Fig. 4](#fig4){ref-type="fig"}, [Fig. 5](#fig5){ref-type="fig"}, [Fig. 6](#fig6){ref-type="fig"} and [Table 4](#tbl4){ref-type="table"}).Parameters for data collectionDescription of data collectionThe collection processes were performed by manual inspections of experts who have Ph. D. degrees in related research fields.Data source locationAs for describing data of past causalities, we referenced past causalities in Japanese textbooks: *Shosetu Sekaishi B* (Se B 304) \[[@bib1]\] and *Sekaishi B* (Se B 301) \[[@bib2]\].\
As for data collected Wikipedia texts, we used following links: <https://www.en.wikipedia/>\*\
where "\*" is replaced by one of from 1 to 1999.Data accessibilityRepository name: Zenodo\
Data identification number: zenodo.3,601,707\
Direct URL to data: <https://doi.org/10.5281/zenodo.3601707>Related research article*Ikejiri, R., Sumikawa, Y.: Developing world history lessons to foster authentic social participation by searching for historical causation in relation to current issues dominating the news. Journal of Educational Research on Social Studies 84, 37--48 (2016). (in Japanese).*<https://doi.org/10.20799/jerasskenkyu.84.0_37>**Value of the Data**•The dataset is useful for training machine learning models as a labeled dataset to predict causality---causality similarity over time.•The dataset can be beneficial for HistoInformatics researchers and history education researchers who are developing new teaching tools to bridge the gap between the past and present, and computer scientists working on temporal data mining including information retrieval.•Once event detections and classifiers are developed by the dataset, they can use them to assign causality categories to other new data. Through this process, the dataset is able to re-examine how accurately categories are defined, identify which categories are (in) dependent of time, or add new categories to the dataset.•We provide the scores of statistical analyses to measure/evaluate dataset quality for estimating which machine learning models can be effectively trained on our dataset. In addition, from these scores, it is able to develop other dataset such as translation to Chinese, adding new temporal events with detail comparisons between our scores and new dataset to identify changes of accuracies.

1. Data Description {#sec1}
===================

The published dataset \[[@bib3]\] (see metadata in [Table 1](#tbl1){ref-type="table"}) consists of seven types of data. The first type includes 138 past causalities in historical_causalities_data.csv file. In this dataset, all causalities include their backgrounds and results. The second type includes the categories that are defined for causalities stored in historical_causalities_categories.csv. This file includes 13 categories: **Reign (Rg)**, **Diplomacy (Dp)**, **War (Wr)**, **Production (Pr)**, **Commerce (Cr)**, **Study (St)**, **Religion (Rl)**, **Literature and Thought (LT)**, **Technology (Tc)**, **Popular Movement (PM)**, **Community (Cn), Disparity (Ds)** and **Environment (En).** The third type includes 900 past event data to support training machine learning models. In the dataset, all events include only descriptions of the events; in other words, the event data excludes both backgrounds and results of the events. The past_events_wikipedia.tsv file contains the descriptions of the 900 past events and their categories. The three files (historical_causalities_regions.tsv, causality_regional_distribution.tsv, and causality_temporal_distribution.tsv) provide additional information of the causalities included in the historical_causalities_data.tsv file. The historical_causalities_regions.tsv includes region where each causality occurred. The other two files (causality_regional_distribution.tsv and causality_temporal_distribution.tsv) include distributions of causalities by regions and centuries, respectively. Last, Statistics.tsv file includes scores of statistical analyses, which are described in "**Statistical Analysis**" section. This file provides all raw scores for estimating which machine learning models are useful for several kinds of applications, e.g., classification and information retrieval (IR) algorithms that can bridge the past and present.Table 1Database files.Table 1File nameContentColumnshistorical_causalities_data.tsvDetail of stored causalities.Causality ID: IDs for causalities.\
Century: Centuries when causalities occurred. BC centuries are represented with minus ("-") in this file.\
Title: Names of the causalities.\
Content: Descriptions of the causalities.historical_causalities_regions.tsvRegions where the causalities occurred.Causality ID: IDs for causalities.\
Regions: Related regions.past_events_wikipedia.tsvDescriptions of past eventsYear: A year of Wikipedia article titles that include the past event\
Categories: Names of categories assigned to the past event.\
Text: Descriptions of the past event.historical_causalities_categories.tsvCategories of the causalities.Causality ID: IDs for causalities.\
Categories: Names of categories.causality_regional_distribution.tsvNumbers of causalities for all regions.The first column includes centuries. BC centuries are represented with minus ("-") in this file.The second column includes\
the number of causalities when occurred in the century of the raw.causality_temporal_distribution.tsvNumbers of causalities for all centuries.The first column includes region names.\
The second column includes the number of causalities where occurred in the region of the raw.Statistics.tsvScores of statistical analyses described in this paperThis file contains all scores of statistical analyses described in the "**Statistical Analysis**" section. This file provides raw data of [Fig. 3](#fig3){ref-type="fig"}, [Fig. 4](#fig4){ref-type="fig"}, [Fig. 5](#fig5){ref-type="fig"}, [Fig. 6](#fig6){ref-type="fig"}.Table 2Statistics of the whole dataset published in this paper.Table 2Number of categories13Number of historical causalities138Number of past events900Ave. Num. of causalities per category10.6Table 3Numbers of the causalities and past events in each category.Table 3RgDpWrPrCrStRlNum. of Causalities46642929511824Num. of Past Events40522937072533183LTTcPMCnDsEnNum. of Causalities261723492220Num. of Past Events403170321432Table 4Intra-category Meta-data and TF-IDF + JS similarities.Table 4RgDpWrPrCrStRlMeta-data10.6%12.0%14.1%12.4%13.6%12.4%13.2%MI0.13950.14220.14780.15430.14330.14730.1650Jaccard0.01750.01790.01760.01810.01760.01870.0180TF-IDF + JS0.95750.95570.95300.95160.95400.96030.9499LTTcPMCnDsEn*Total*Meta-data12.4%12.2%13.5%11.5%12.5%12.8%*12.5%*MI0.14860.13960.14470.13940.14010.1511*0.1463*Jaccard0.01840.01860.17690.01860.18370.0185*0.0552*TF-IDF + JS0.95360.95840.95460.95930.94880.9540*0.9546*

2. Experimental design, materials, and methods {#sec2}
==============================================

Causality Data Collections {#sec2.1}
--------------------------

The 138 past causalities were created by authors in three steps. First, we collected over 700 past causalities by referencing well-known Japanese high-school textbooks: Shosetu Sekaishi B (Se B 304) \[[@bib1]\] and Sekaishi B (Se B 301) \[[@bib2]\]. Second, we selected the causalities if they could be useful for considering solutions for present social issues. Finally, we described each causality in three contexts: background including problems, solution methods, and their results.

Category Definition {#sec2.2}
-------------------

The causality categories are defined to organize the causalities with the useable historical framework \[[@bib5]\] as described in Ref. \[[@bib4]\]. In Ref. \[[@bib5]\], Lee claims that causalities over different times can be bridged if they belong to the framework because the framework is an overview of the long-term patterns of change and not a mere outline story skimming a few peaks of the past. Under this idea, the category definition processes comprised 2 steps. First, we reviewed Encyclopedia of Historiography \[[@bib6]\] to define categories for connecting past and present causalities. In the review process, we listed all the main topics from the encyclopedia and subsequently selected a topic only if it included the long-term patterns of change. Second, we evaluated if each topic included causalities independent of time. As we extracted causalities from history textbooks, we divided them into three temporal periods: ancient, medieval and modern periods. If a topic included causalities from the three temporal periods, we used it as a category in the dataset. Moreover, we added some new categories in the dataset if we found new topics that included causalities from the three temporal periods. Finally, we defined the 13 categories described in **Data Description** section.

Event Data Collections {#sec2.3}
----------------------

The 900 past event data were crawled from Wikipedia articles whose tiles were the years from 1 to 1999, for example, <http://en.wikipedia.org/1>. The collection process was as follows: 1) All events were crawled from yearly Wikipedia articles. 2) It was manually reviewed whether the crawled events could be useful to consider any solutions for present social issues. 3) At most, 50 events per century were randomly sampled to cover a wide range of durations.

Basic Statistics {#sec2.4}
----------------

[Table 2](#tbl2){ref-type="table"}, [Table 3](#tbl3){ref-type="table"} summarize the statistics of the entire published dataset and the number of causalities for each category, respectively.

[Fig. 1](#fig1){ref-type="fig"}, [Fig. 2](#fig2){ref-type="fig"} plot distribution of the numbers of causalities by region and centuries. These figures help us to understand tendency of the published dataset because temporal and spatial features are the most important features of history. [Fig. 1](#fig1){ref-type="fig"} plots the number of causalities per century. Naturally, the distribution curve increases near the present. This indicates that the closer the causalities are located to the present on the temporal axis, the greater is their usability for considering solutions for present issues. [Fig. 2](#fig2){ref-type="fig"} plots the distribution of the number of causalities where they occurred. We can see that most causalities occurred in China and Europe, as they have long-term histories.Fig. 1Numbers of causalities per century. This figure represents B. C. years as hyphen ("-").Fig. 1Fig. 2Numbers of causalities per country/region.Fig. 2

Statistical Analysis {#sec2.5}
--------------------

In addition to the basic statistics, the published dataset provides scores of similarity between data points and statistics of clusters to help to train machine learning models. The provided scores are results of the following five analyses.1.Calinski and Harabasz (CH) \[[@bib7]\]. This measure estimates how close all data to each other in a cluster and how far data in different clusters locate. Thus, the higher score of this measure indicates the high quality of the given clusters. The formal equation is defined as follows:$$CH\left( k \right) = \ \frac{\left( {n - k} \right)\ B\left( k \right)}{\left( {k - 1} \right)\ W\left( k \right)}$$where *B*(*k*) and *W*(*k*) are intra- and inter-cluster sums of squares for *k* clusters, respectively, and *n* is the number of clustered data.2.Mutual information (MI). This measure evaluates the similarity of two categories $A$ and $B$ as an information-theoretic approach. Let *P*(*a*) and *P*(*b*) are marginal probabilities, and *P*(*a, b*) is a joint probability. MI calculates volumes of information a given set generates about the other set. This is done by the following equation.$$MI\left( {A,\ B} \right) = \ \sum\limits_{b \in B}{\sum\limits_{a \in A}{P\left( {a,\ b} \right)\log\frac{P\left( {a,\ b} \right)}{P\left( a \right)\ P\left( b \right)}}}$$

The provided scores of this dataset are generated from adjusted MI (AMI) \[[@bib8]\] that is a variant of MI defined as follows:$$AMI\left( {A,\ B} \right) = \ \frac{MI\left( {A,\ B} \right) - E\left( {MI\left( {A,\ B} \right)} \right)}{max\left( {H\left( A \right),\ H\left( B \right)} \right) - E\left( {MI\left( {A,\ B} \right)} \right)}$$where $\left. E\left( MI\left( {A,\ B} \right) \right. \right)$ is the expected MI between two given categories, $max$ is a function that returns the largest value among given values, and $H\left( A \right)$ is the entropy of category $A$.3.Jaccard index (Jaccard). This measure employs an assumption that if two sets $S_{A}$ and $S_{B}$ for two categories $A$ and $B$ have many common data, then the two sets are similar to each other. As the size of given sets affects the common numbers, this measure normalizes the score by taking account of the total sizes of given two sets. In other words, if a given set is huge compared with other sets, the huge set tends to include several elements of other sets. This idea is represented as follows:$$Jaccard\left( {S_{A},S_{B}} \right) = \frac{\left| {S_{A} \cap S_{B}} \right|}{\left| {S_{A} \cup S_{B}} \right|}$$4.TF-IDF + Jensen--Shannon (JS) divergence. This is an entropy-based measurement; the lower this score, the more similar the two given probability distributions are. As our dataset includes texts written in natural language, we first apply TF-IDF to convert the text into numbers. Let $w$, $d$ and $W\left( d \right)$ are a word, a document, and a word set of $d$. The TF-IDF estimates the importance of $w$ in $d$ by counting the numbers of occurrences of the word in the document and by the numbers of documents including the word. Once all data can be converted to vectors whose elements are scores of the importance of the words, the similarity between two data can be measured by JS divergence that is an extension of Kullback---Leibler (KL) divergence. These approaches are defined as follows:$$TF - IDF\left( {w,\ d} \right) = \ tf_{w,\ \ d}\frac{\left| D \right|}{\left| \left\{ {d^{'} \in D\ \text{|}\ w_{i} \in W\left( {d'} \right)} \right\} \right|}$$$$KL\left( {A,\ B} \right) = \ \sum\limits_{a \in A,\ b \in B}{a\ \log\frac{a}{b}\ \ }$$$$JS\left( {A,\ B} \right) = \ \frac{1}{2}KL\left( {A,\ M} \right) + \ \frac{1}{2}KL\left( {M,\ B} \right)$$where $M$ is $\frac{1}{2}\left( A + B \right)$.5.Meta-data similarity. This measure counts the number of common categories shared by two causalities. Similar to the Jaccard index, the higher the score, the more similar the two causalities are. Thus, given two causalities, the measure is represented as the sum of the common categories. This is formally defined as follows:$$Meta\left( {FV_{i},\ FV_{j}} \right) = \sum\limits_{k = 1}^{FV}{AND\left( {FV_{ik},\ FV_{jk}} \right)}$$where *AND* is the logical AND. It is 1 if both operands are 1; otherwise, it is 0. This measure considers the feature vectors ($FV_{i}$ and $FV_{j}$) of two causalities that are defined from the categories of the two causalities. If causality $C_{i}$ has the *k*th category, then $FV_{ik}$ is 1; otherwise, it is 0.

We applied the above measures for all combinations of causalities within each category (intra-category) and within two different categories (inter-category), which are described as follows:1.Intra-category Similarity. This similarity represents the average score of similarity between all combinations of two causalities in a category.2.Inter-category Similarity. This similarity represents the average score of similarity between all combinations of two causalities from two different categories.

Scores of Statistical Analysis {#sec2.6}
------------------------------

The CH score for the past causality data was 1.0829. This indicates that the intra-cluster and inter-cluster sums of squares for the *k* clusters are almost the same. [Table 4](#tbl4){ref-type="table"} shows all scores of all intra-category measurements. Overall, these scores indicate that all texts in the same category are not similar to each other. These scores indicate that the published dataset covers several kinds of causality topics. If it is necessary to train machine learning models only on the causality texts, it is better to use simple IR algorithms, for example, the query word matching method, or to employ transfer learning using the categorized past events.

[Fig. 3](#fig3){ref-type="fig"} shows all scores of the inter-category meta-data similarities on the causality data. Three combinations of two categories, **Rg**---**Dp**, **Cr**---**St** and **Ds**---**PM** contain more common categories compared to other combinations.Fig. 3Inter-category meta-data similarity.Fig. 3

[Fig. 4](#fig4){ref-type="fig"} plots the MI scores for all combinations of categories on the causality data. This figure indicates that three categories (**St**, **Rl** and **LT**) are more similar to each other compared with other category combinations. [Fig. 5](#fig5){ref-type="fig"} plots the Jaccard index for the inter-category analysis. Similar to the scores of MI, three categories (**St**, **Rl** and **LT**) are more similar to each other compared to others. However, the Jaccard similarity scores between the three categories are lower than the MI scores. [Fig. 6](#fig6){ref-type="fig"} shows the TF-IDF + JS scores. Two combinations of categories are similar to each other. The scores for the **St**, **Rl** and **LT** categories are smaller compared to other combinations. In addition, the **Dp**, **Wr**, **Pr** and **Cr** categories have relatively lower scores compared to other combinations except the combination of **St**, **Rl** and **LT**. Thus, these four categories are more similar to each other compared with other category combinations.Fig. 4Inter-category MI score.Fig. 4Fig. 5Inter-category Jaccard scores.Fig. 5Fig. 6Inter-category TF-IDF + JS scores.Fig. 6

Summary of the Statistical Analyses {#sec2.7}
-----------------------------------

All statistical analyses for causalities indicated that the similarities between intra- and inter-category data tended to be low. Thus, if it is necessary to use only past causality data in machine learning study such as IR specialized for history, it is better to use simple techniques, e.g., simple word-based pattern matching and counting common categories. In contrast, if it is able to use both past causality and event data together, using more sophisticated machine learning models such as SVM, naive Bayes classification, and random forests is a good choise as the published data includes 1038 categorized data.

Conflict of Interest {#appsec3}
====================

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data {#appsec1}
==============================

The following is the Supplementary data to this article:Multimedia component 1Multimedia component 1

This work was supported by JSPS KAKENHI Grant Number 26750076, 17K12792, and 19K20631.

Supplementary data to this article can be found online at <https://doi.org/10.1016/j.dib.2020.105185>.
