Measuring Sustainability Reporting using Web Scraping and Natural Language Processing Keywords: web scraping, NLP, natural language processing, SDG, sustainable development goals, sustainability, reporting, indicator In September 2015 the United Nations stipulated its requirements for Sustainable Development Goals (SDG’s). The Goals are being followed-up and reviewed using a set of global indicators, which result from the aggregation of the national level indicators produced by each member state. However, there are many indicators for which national level statistical estimates are not currently produced. This abstract highlights some proof-of-concept research using Web-based data for an indicator relating to SDG Target 12.6 which is encourage companies, especially “To large and transnational companies, to adopt sustainable practices and to integrate sustainability information into their reporting cycle” [1]. The indicator proposed is "Number of companies publishing sustainability reports, by turnover band, geography, national or global company, sector and number of employees”. This document outlines the steps taken to develop a web scraping program able to collect sustainability information from websites of a sample of the 100 largest UK private companies (ranked according to sales), followed by the use of natural language processing (NLP) techniques to process and extract additional insights from the data collected. Web scraping is a technique to extract data from websites without the need of a user interaction. The web scraping program, developed in Python, accesses all the 100 companies’ websites provided and looks for pages that contains text relating to sustainability information. Text analysis on the HTML content of the selected pages is used to extract additional insights. The scraper For each company, the scraper navigates through the company website, accessing internal link. While recursively traversing websites, the scraper flags only the pages that suggest sustainability content, i.e. at least one of the predefined keywords is found in the URL address of the page or the text of the hyperlink leading to the page. The list of keywords, manually chosen, is as follows: csr, environment, sustainab, responsib, Once a page containing a keyword is found, it is cleaned before being saved in footprint. a database. A high level overview of the architecture of the scraper is shown in Figure 1. Figure 1. High overview of the web scraping program
Measuring Sustainability Reporting using Web Scraping and Natural Language Processing Keywords: web scraping, NLP, natural language processing, SDG, sustainable development goals, sustainability, reporting, indicator In September 2015 the United Nations stipulated its requirements for Sustainable Development Goals (SDG’s). The Goals are being followed-up and reviewed using a set of global indicators, which result from the aggregation of the national level indicators produced by each member state. However, there are many indicators for which national level statistical estimates are not currently produced. This abstract highlights some proof-of-concept research using Web-based data for an indicator relating to SDG Target 12.6 which is “To encourage companies, especially large and transnational companies, to adopt sustainable practices and to integrate sustainability information into their reporting cycle” [1]. The indicator proposed is "Number of companies publishing sustainability reports, by turnover band, geography, national or global company, sector and number of employees”. This document outlines the steps taken to develop a web scraping program able to collect sustainability information from websites of a sample of the 100 largest UK private companies (ranked according to sales), followed by the use of natural language processing (NLP) techniques to process and extract additional insights from the data collected. Web scraping is a technique to extract data from websites without the need of a user interaction. The web scraping program, developed in Python, accesses all the 100 companies’ websites provided and looks for pages that contains text relating to sustainability information. Text analysis on the HTML content of the selected pages is used to extract additional insights. The scraper For each company, the scraper navigates through the company website, accessing internal link. While recursively traversing websites, the scraper flags only the pages that suggest sustainability content, i.e. at least one of the predefined keywords is found in the URL address of the page or the text of the hyperlink leading to the page. The list of keywords, manually chosen, is as follows: csr, environment, sustainab, responsib, footprint. Once a page containing a keyword is found, it is cleaned before being saved in a database. A high level overview of the architecture of the scraper is shown in Figure 1. Figure 1. High overview of the web scraping program
x_abstract_36.docx