This tutorial shows how we can use different aspects of Python (i.e. dictionaries, lists, and regular expressions) together to solve different issues. It also shows how we can use Python to link the relationships in the MeSH file, making it easier to understand its hierarchy and structure.
Before moving ahead with this tutorial, you might be wondering what we mean by MeSH. So let's start by defining this term first, and then go into a bit more detail on its structure.
What Is MeSH?
MeSH is an acronym for Medical Subject Headings. It is considered the U.S. National Library of Medicine's controlled vocabulary (thesaurus), which gives uniformity and consistency to the indexing and cataloging of biomedical literature. MeSH, a distinctive feature of MEDLINE, is arranged in a hierarchical manner called the MesH Tree Structure, and is updated annually.
MeSH is thus a nomenclature of medical terms available from the U.S. National Library of Medicine, that aims to create new knowledge by exploiting the relationships among terms that annotate the biomedical literature.
People searching MEDLINE/PubMed and other databases make use of MeSH to assist with subject searching. The National Library of Medicine (NLM) indexers use MeSH to describe the subject content of journal articles for MEDLINE. Catalogers use MeSH to describe books and audiovisuals in the NLM and other library collections. So MeSH can be used for numerous tasks involving indexing, tagging, searching, retrieving, analyzing, coding, merging, and sharing biomedical text.
MeSH File Structure
MeSH descriptors are organized into 16 categories:
- A: anatomy
- B: organisms
- C: diseases
- D: drugs and chemicals
- E: analytical, diagnostic and therapeutic techniques and equipment
- F: psychiatry and psychology
- G: phenomena and processes
- H: disciplines and occupations
- I: anthropology, education, sociology, and social phenomena
- J: technology, industry, agriculture
- K: humanities
- L: information science
- M: named groups
- N: health care
- V: publication characteristics
- Z: geographicals
You can find more information about the categories from the U.S. National Library of Medicine. As we can see, each category is further divided into subcategories. This structure is, however, not considered an authoritative subject classification system, but rather as an arrangement of descriptors for the guidance and convenience of people who are assigning subject headings to documents or are searching for literature. It is thus not an exhaustive classification of the subject and contains only the terms that have been selected for inclusion in this thesaurus.
Here's some more information on the MeSH Tree Structures:.
Downloading a MeSH File
For the purpose of this tutorial, we need a MeSH file to work with in Python. You can find MeSH file on the NLM download site.
Let's go ahead and download the latest ASCII MeSH file. We can first go to the MeSH FTP Archive:, and then choose the
2017 directory. In the
asciimesh/ directory, you will find three
.bin files:
c2017.bin,
d2017.bin, and
q2017.bin. Let's download
d2017.bin. You can download the file from: (27.5 MB).
Linking Terms to Numbers
Let's jump into the core of this article. What we are trying to do is read a MeSH file (i.e. the
.bin file you just downloaded), browse through the entries, find all the MeSH numbers for each entry, and list the terms along with their relevant numbers.
The first thing we would normally do is read the
.bin file, as follows:
meshFile = 'd2017.bin' with open(meshFile, mode='rb') as file: mesh = file.readlines()
Notice that we have used the
rb mode, meaning that we are reading binary with no line-break translation.
We also need to define an output file where we would store the results (output):
outputFile = open('mesh.txt', 'w')
At this point, we want to check the lines that start with
MH = (MeSH term) and
MN = (MeSH number). I shouldn't do this now, but will show you a snapshot of the MeSH file to have some idea of the structure and to remove any confusions (
MH and
MN are surrounded by red rectangles, respectively).
To check lines that start with
MH = and
MN =, we need to use regular expressions. So, if we want to check the lines that start with
MH = followed by any characters, we would do as shown in the code below (I'll get to what
line is in a moment). Notice that I have used
b instead of
r for the regular expression, since we are applying the pattern on a byte object and not a string object, so we should use a byte pattern.
import re meshTerm = re.search(b'MH = (.+)$', line)
The same thing would apply for the MeSH number, but this time for lines starting with
MN =.
Coming back to
line, this refers to the lines in the MeSH file. So we would be walking through the file line by line, looking for the MeSH terms and numbers. As you can see from the above MeSH file snapshot, the MeSH term comes before the MeSH number. So, in our code, the MeSH number will always be the number corresponding to the previously captured MeSH term. We will thus do the following:')
Let's go through the above code step by step. If we look at the regular expression
MH = (.+)$, this is basically telling us to find the literal
MH = followed by at least one character. (
.) means any character, and
+ means that it has to be one or more characters, and return everything to the end of the line (
$).
The parenthesis around
.+, that is
(.+), is a capture group so we can retrieve the result. So, for the MeSH term surrounded by a red rectangle in the above snapshot, the retrieved term will be
Calcomycin. The reason we are using if-statements is that some lines will neither start with
MH = nor
MN =.
For the captured MeSH term and MeSH number, we create a new key-value pair for a dictionary object, as shown in this line of code:
numbers[str(number)] = term.
It is important to note that a single MeSH term might have more than one MeSH number. So we concatenate every new MeSH number with the relevant term into a string, as shown in this portion of the code:
if term in terms: terms[term] = terms[term] + ' ' + number.decode('utf-8') else: terms[term] = number.decode('utf-8')
Thus in this case we will be having a dictionary object with key-value pairs that consist of a MeSH term as the key, and the concatenation collection of all corresponding MeSH numbers as the value.
What we want to do now is list the different keys (terms), and have the relevant values (numbers) listed under the relevant term. To list the different terms, we do the following:
meshNumberList = [] meshTermList = terms.keys() for term in meshTermList: item_list = terms[term].split(' ') for phrase in item_list: meshNumberList.append(phrase)
Finally, we will list the term and its relevant numbers as follows:
used_items = set() for item in meshNumberList: if numbers[item] not in used_items: print(numbers[item], '\n', item, file=outputFile) used_items.add(numbers[item]) else: print(item, file=outputFile)
Before showing the output of the program, let's put it all together.
Putting It All Together
In this section, I will show you what our full Python program that links the MeSH term to its numbers looks like:
import re terms = {} numbers = {} meshFile = 'd2017.bin' with open(meshFile, mode='rb') as file: mesh = file.readlines() outputFile = open('mesh.txt', 'w')') meshNumberList = [] meshTermList = terms.keys() for term in meshTermList: item_list = terms[term].split(' ') for phrase in item_list: meshNumberList.append(phrase) meshNumberList.sort() used_items = set() for item in meshNumberList: if numbers[item] not in used_items: print(numbers[item], '\n', item, file=outputFile) used_items.add(numbers[item]) else: print(item, file=outputFile)
Output
You can download the output from Dropbox (1.77 MB). Taking a sample of the output as shown below, we can see how a MeSH term (
Pterygopalatine Fossa) is listed with its MeSH numbers that are grouped immediately underneath.
Pterygopalatine Fossa A02.835.232.781.670 A02.835.232.781.750 A02.835.232.781.750.150 A02.835.232.781.750.165 A02.835.232.781.750.400
Conclusion
The tutorial showed how we can use different aspects of Python (i.e. dictionaries, lists, and regular expressions) together to solve different issues. It also shows how we can use Python to work with MeSH files for linking some parts of this complex file in a way that makes it easier to understand its hierarchy and structure, as we did here by linking the MeSH term to its relevant MeSH numbers.
Envato Tuts+ tutorials are translated into other languages by our community members—you can be involved too!Translate this post
