Design and Implementation of Interactive English Translation System in Internet of Things Auxiliary Information Processing

Information technology has penetrated into all aspects of human life. Nowadays, with the rapid development of science and technology, information technology has gradually become the cornerstone of the development of other technologies. The Internet of Things is an important part of the new generation of information technology. Language is the medium of communication between people. Driven by economic globalization and the development of the Internet, information is growing rapidly, and there are more and more exchanges and exchanges between countries. The emergence of high-e ﬃ ciency and high-economic machine translation solves these di ﬃ culties, and the interactive English translation system is the current research hotspot, which is intended to improve the output translation quality of the English translation system. The main work of this paper is to analyze the existing interactive machine translation technology, especially the interactive machine translation based on a phrase model, using the Internet of Things as a knowledge source. According to the characteristics of segment analysis and human-computer interaction mechanism, from the network, a wealth of information are available from open sources. In this paper, on the basis of the segment analysis, the human-machine cooperation translation strategy of human-machine cooperation with complementary human-machine advantages was discussed, and the system designed was veri ﬁ ed. It is proved that the system has high performance in improving the accuracy and recall rate of machine English translation. Compared with the existing English translation system, the accuracy has improved by more than 20% in the case of fewer iterations, and in the case of 90 iterations, the accuracy can improve by 100%.

Introduction
A new generation of information technology is booming, and the Internet of Things technology is one of the emerging information technologies in recent years. Its technical characteristics are based on information perception technologies such as radio frequency identification and sensors and are based on Internet technology. Through information perception, the interaction and fusion of the physical world and the information world are realized, and intelligent computing technology is used to process and analyze the perceived information, thereby realizing intelligent control and decision support. In today's increasingly frequent global communication, communication between people inevitably requires mutual translation of various languages. Manual translation is costly and inefficient, making it difficult to meet large-scale translation needs. Therefore, people's demand for automatic translation technology is increasing day by day, and the development of machine translation technology is the general trend [1]. Interactive machine translation (IMT) is an important research topic in the field of computer-aided translation. In the process of translation, the system and users learn from each other to instruct the computer to decode, so as to improve the quality of the output translation of the machine translation system. Compared with fully automatic machine translation, interactive machine translation makes full use of human linguistic knowledge to guide decoding, which can improve the quality of translation and enhance the practicability of translation. Compared with human translation methods, the addition of machine translation systems can provide translators with initial translations, reduce the workload and cognitive burden of users, and improve translation efficiency. The interactive machine translation method provides an interface for mutual learning and communication between the translator and the machine translation system and combines the high efficiency of the machine translation system with the high accuracy of the translator's translation, which not only reduces the translator's workload but also improves the translation quality. With the continuous deepening and development of technology, how to effectively use existing technologies such as machine translation technology and cloud computing technology to improve the overall level and production efficiency of China's translation industry has become a focus of attention of enterprises, society, and the country [2].
In this paper, the interactive machine translation technology based on segment analysis, which takes the Internet as the knowledge source, has improved the quality and efficiency of machine translation from the perspective of expanding the scope of knowledge acquisition and adopting interactive strategies. When selecting the corpus in the test set as the analysis object of this paper, the translation platform performs poorly under certain error types, with 120 discourse errors. When analyzing the ontology errors in English translation errors, the frequency of ontology errors accounted for a low proportion of the total error frequency, but the original text recognition and sentence segmentation errors were very concentrated, much higher than other error types, with a frequency of 20 and 23. The analysis of semantic errors in segfaults shows that except for the frequent occurrence of terminology errors, other error types are basically stable below 4 times. The accuracy and recall rate of machine English translation using the system in this paper is high, and it can reach 100% in 90 iterations with good performance.
Today is a new time of digitalization and information technology. As a result, the new informatics featuring the Internet and computers have penetrated into all areas of our lives and influenced our consumption views and lifestyles. As far as translation teaching is concerned, the traditional translation teaching mode cannot satisfy the demands of contemporary education. Qiao and Wang analyzed the characteristics of m-teaching and constructed a new paradigm of interactive mobile translating instruction from the viewpoint of structuralism. The new model is initially demonstrated by using three kinds of the popular social interactive tools in China, WeChat, QQ, and Weibo, as examples. The results show that this Internet+-based approach can enhance language translation instruction [3]. To enhance the precision of English interpretation, decrease the rate of error in interpretation outcome, and improve the accuracy of interpretation, Leida Wu and Lianguan Wu proposed a design of architectures for business English interpreting based on speech detection and radio transmission. According to the functional division of the overall system design, the speech collection block, speech treatment block, and external electric module are devised on the basis of functionality request. Simultaneously, a business English interpretation database was built to meet the individual demands of customers by making use of radio transmission technique. An optimized translation model is also applied for translation error correction and intelligent checking to increase the accuracy and accuracy of translations. The results of the trial show that the system has a high correction rate and error rate, and the translating results have certain level of stability, which fully validates the system's validity and application value but has no wide application [4]. With the advancement of information technology and quality training, to enhance the comprehensive quality and subjective motivation of students, Wu has explored some new teaching methods and teaching methods. The interactive mode of English translation based on data mining is analyzed. The implementation of interactive teaching mode in English translation teaching activities has not only met the individual needs of students but also improved the effectiveness of teaching and accelerated the construction of knowledge and skills in the process of English translation [5]. Machine translation (MT) by itself is often not sufficient to generate good quality results, so humans often intervene in the translating procedure to enhance the machine translation output. One type of interference is typically postediting (PE), in which manual translators correct mistakes in the machine translation outputs. Another is interactive translation prediction (ITP), which deals with the machine translation process that suggests interpreter translations that can be acceptable or rejectable. Later, the machine translation-oriented system uses these movements to suggest new, correct ones to them. Rebecca presented the results of an empirical study of the latent neural machine translation system in ITP (NITP) on translation related to translation related creativity. The results showed that more than half of the professional translators in this study used NITP to translate faster than PE and the most favored NITP over PE [6]. The translation system design of English has been a hot research topic, but not many studies have been conducted on the interaction direction of English translation system design.
Recent technological developments and innovations have improved the way people live with intelligent app, sensors, and mobile telecommunication nets. With respect to all of them, the Internet is the main backbone, allowing the access and delivery of essential pieces of info handling through the Internet of Things (IoT). IoT holds support for multidisciplinary applications and is an energetic physical entity in fields of engineering, surgery, and business. Raj developed a novel type of message handling system on an IoT platform through a robust medical surveillance system. With the proposed architecture, the efficient utilization of big data in IoT context is analyzed to achieve minimum delay in a real-time environment. The performance of the proposed design is compared using conventional models, and experiments are carried out to verify the excellent capabilities of the suggested approach in terms of the transfer and storage cost functions, f -measure, flexibility, and idiosyncrasies [7]. The development of the Information of Things (IoT) has enabled technology to share physical education by connecting cost-effective disparate devices and of digital-based programs into an effortlessly accessible and uncontrolled setting. Li et al. proposed IoT-assisted physical activity monitor devices to track students' physical activity and enhance outcomes. Management skills enable students 2 Wireless Communications and Mobile Computing to structure and accelerate physical activity in a healthy way. In a further step, the link of monitoring ability, which is an essential element of bodily activity, to physical activity was investigated. The system collects baseline information from IoT-based wearing items that interact with the data in real time through virtual devices. The IoT network includes activity from multiple units and detects a person's heartbeat and body weight. Experimental results show that the approach enables very good physical activity monitoring results compared to a conventional system [8]. In recent years, the highly prolific Industrial Internet of Things (IIoT), consisting of nodes of heterogeneous resource-constrained IoT, has received significant interest from both academic and industrial communities. Based on the emerging edge computing paradigm and the novel reliable ionic overlay (CIC) model, Wang et al. investigated how to relocate redundant IoT edge nodes to provide timely and reliable information overlay services to mitigate the auxiliary energy balance while extending the network lifetime, which is the CIC-based IoT edge node relocation (CICENR) problem. To effectively solve the CICENR problem, a load-assisted energy-balanced IoT edge node relocation method (CIC-OAEBA) and another CICbased direct replacement method (CIC-DRA) are proposed.
Experimental results show that this method significantly outperforms other peer-to-peer approach in terms of response time, energy efficiency, and especially in terms of network lifetime and coverage performance [9]. For the related work section, these studies provide a detailed analysis of English translation and IoT information processing techniques. It is undeniable that these studies have greatly promoted the development of the corresponding fields. We can learn a lot from methodology and data analysis. However, the research on the design of interactive English translation system using information technology is relatively few and not thorough enough, and it is necessary to fully apply these technologies to the research in this field.

Method of Interactive English Translation System in Internet of Things Auxiliary Information Processing
In recent years, the concept of Internet of Things (Internet of Things) has become very popular. Its core concept is based on radio frequency identification (RFID), Electronic Product Code (EPC), and the Internet. A tangible Internet capable of sharing global project information in real time was established. It is an extension and expansion network based on the Internet. It is a huge network formed by combining various information sensing devices with the network to realize the interconnection of people, machines, and things at any time and place [10]. The Internet of Things is actually a combination of information and industry, which connects real matter and virtual matter and connects objects and the Internet through different information perception devices to exchange, communicate, and process information to intelligently identify, locate, track, monitor, and manage [11,12]. The basic characteristics of IoTs can be summarized as overall perception, reliable transmission, and intelligent processing. Information processing mainly includes signal preprocessing, processing, postprocessing, feature selection, and extraction. Information identification is to identify, compare, classify, and judge the information [13]. On this basis, an association model based on one-to-one correspondence is introduced to optimize the input feature information, thereby realizing the next level of communication [14].
To make things intelligent, interact with people, and interact with things, there must be strong technical support, including intelligent control, human-computer interaction, intelligent information processing, high-performance cloud computing platforms, data storage management systems, databases, enterprise resource planning, and decision support [15].
Interactive machine translation (IMT) is a technology that improves the conversion efficiency between natural languages through machine translation and human-computer interaction [16]. The basic idea of interactive machine translation is to allow users to perform manual intervention and guidance in the process of translation, so as to obtain higher-quality translations. Interactive translation systems are characterized by interactivity. Every time the user interacts with the system, i.e., after each word (or letter) is introduced, the system generates new translation hypotheses. In interactive machine translation (IMT), in each interaction, a machine translation engine is first used to predict a particular source sentence, which is then accepted, modified, or ignored by the translator (continue input) [17]. This cycle repeats until the final translation. Although the existing computer interaction translation has achieved certain achievements, it also has its shortcomings.
2.1. Instance-Based Machine Translation. For example-based machine translation, the main source of knowledge is the example translation library of the parallel corpus, which consists of mainly two fields of interest. One is the sentence in the source language, and the other is the counterpart sentence in the destination language. When a source languagebased sentence is input, the translation system finds the most similar source-language sentence in the library, mimics the respective target-language sentence, and produces the respective translations [3].
The whole translating process is for finding and copying results. It only makes a comparison of the source language sentences and does not carry out analysis.

Statistics-Based Machine Translation.
Statistical machine translation is to perform mathematical statistics and analysis on a large number of translation bilingual materials through machine learning for translation. That is, from the source language to the translation into the target language, the whole process is a problem of mathematical probability and statistics. Each sentence in the target language may be the translation of any sentence in the source language, but the probability is different. Therefore, machine translation is to find the target language sentence with the highest probability [18].
The current multilingual machine translation system is mainly divided into three parts, among which the data preprocessing module and the model building module are the focus of the research. Model evaluation and microservice modules are used to evaluate the overall model performance and translation generation fluency. It is finally displayed with the front-end interface. Figure 1 shows the overall architecture of the English-to-English machine translation system.
In terms of architecture, it consists of three parts: user interface, multilingual neural translation model, and data processing module. The human-computer interface usually refers to the part that is visible to the user, where the user communicates with the system through the humancomputer interface and performs operations. Only by integrating the concept of human-computer interaction and the concept of people as the main body into the design can it be favored by consumers. Human-computer interaction design can directly affect the user experience. The overall design structure of the interactive English-Chinese machine translation system based on segment analysis with the Internet as the knowledge source is shown in Figure 2 [19].
Given a source language sentence l, the purpose of statistical machine translation methods is to find a target language m such that the conditional probability pðm | lÞ reaches the maximum value [20]. The modeling of phrase machine translation mainly adopts the logarithmic line model. In the logarithmic line model, the conditional probability pðm | lÞ can be expressed by Among them, T is the feature dimension, and γ and h are the feature value (model score) and its corresponding weight, respectively. In the phrase-based statistical machine translation system, translation model features, language model features, lexical ordering model features, counting features, and translation distortion features are mainly used [21].
The goal of machine translation is to learn translation knowledge from large-scale bilingual parallel corpora for automated translation. On this basis, the goal of machine translation to solve the optimal translation can be written as formula (2), which is the optimal translation to be solved.
In practical applications, a machine translation model is composed of multiple models (features). Using a log-linear model to model the above formula can combine all the models. The formula is as follows: Among them, f i ðm, lÞ is represented as a model, N is the number of all models, and γ i is the weight of the model.
Using the log-linear model can adjust the contribution of different models by assigning different weights to the submodels to improve the translation quality of the translation. At the same time, the structure of the log-linear model does not limit the number of submodels, and submodels can be added, deleted, or modified at any time according to needs. [22].
Interactive machine translation is that on the basis of machine translation, the translation system searches for qualified translation suffixes according to the source language and the prefix constraints specified by the translator and makes appropriate changes to the machine translation framework to obtain the framework of interactive machine translation. Its formula is shown in (4) [23].
Meanwhile, Formula (4) Prðm p Þ in Formula (6) is input by the translator, not related, so it is omitted. Compared with statistical machine translation, the prefix-based human-computer interaction mechanism enables translators to participate in machine translation and decoding, so that translators can fully utilize the advantages of language knowledge, and the accuracy of translators and the efficiency of machine translation can be combined well.
SWECCL is a typical learner corpus. The full name of SWECCL is China English Corpus, which contains a large number of learning tools and research materials and is of great help to English two-way translation. The translation function of this system is centered on the main function of the system. It uses the artificially input vocabulary and the vocabulary in the SWECCL corpus to perform the correlation operation and uses the weight function method to calculate the similarity between the translated words and the SWECCL corpus. And on this basis, the information required by the user is given.
Assuming that the user manually enters the vocabulary n and the vocabulary in the SWECCL corpus is m, the basic form of calculating the similarity between two words is as follows: Since the term n has been defined, the weight of c can be ignored in the calculation. In the formula, c represents the term, εðc, mÞ represents the weight of c in m, and εðc, nÞ represents the weight of c in n. c's power is given as

Wireless Communications and Mobile Computing
The weighted function that combines the terms is carried into Equation (8) to obtain Equation (9).
In this equation, εðsiÞc denotes the value of si weight of the knot opposite to entry c, which denotes the importance of this entry. In order to prevent the entry importance from influencing the likeness computing, log is used to process si, and the basic equation for the likeness computing of the weighted network-based message translation module is obtained.
According to the above formula, the vocabulary size manually entered by the user in the SWECCL corpus can be obtained and compared with the similarity in the SWECCL corpus. The SWECCL corpus is used as the research object, and the SWECCL corpus is applied to the system data.
On this basis, a new feature extraction method is proposed to apply the optimal context mapping to translations to achieve the standard extracting of characteristic situations, and the optimal context is depicted by the Semantic Ontology Mapping Model. Assume that there are N different translating contexts in the translation process, consisting of G -level semantic conversions. The number of translation contexts is N i ði = 1, 2, ⋯, GÞ, the probability of G-level semantic transitions is X i = fX i1 , X i2 , ⋯, X iN g, and X ij = fi = 1, 2, ⋯, K ; j = 1, 2, ⋯, N i g is the result of an n-dimensional vector in one direction. In the course of definition, the fundamental standard context of translation can be implemented.
In the formula, ϑ i is the context of translation-ready terminology translation, and the best context ϑ is chosen by the procedure.
The matrix A w of the context of nonsemantic interpretation and the context matrix A B of the proper translation of context of meaning are separately calculated as shown below.
Let σ be the best semantic background of the contextual association matrix A T w A B and f be the measure of semantic contextual association; then the numerical value of ϑ can directly reflect the mapping of the association process. The optimal situation of the extracted context is A T w A B ; then the semantics of the characteristics in the optimized situation can be denoted by β. The maximum number of optimal translation contexts in the semantic context association matrix RðR ≤ K − 1Þ is K − 1.
Intuitively, the higher the quantity of words cooccurring in both words, the larger the similarity, and the similarity calculation on the basis of cooccurring words is based on this. The resemblance here is not only influenced by the number of cooccurring words but can also be measured by the total number of words contained in the sentence, i.e., the sentence length, which can be more intuitively expressed by the following equation: Among them, InterðI, OÞ is the number of cooccurrence words between the input sentence I and the retrieved sentence R, and UnionðI, OÞ is the number of words in the set composed of the words of the two sentences.
Computer-aided translation has become a hot research topic in the current machine translation field because it cleverly avoids the bottleneck problem in linguistics. It uses database and other methods to store pairs or more source language sentences and translations that have been translated into translation memory. When the user is translating, the auxiliary translation system automatically analyzes and searches the content according to the current input sentence in the source language and provides the translation that is closest to the input sentence in the memory to the user for reference. In special cases, when the input sentence can be completely matched with the sentence stored in the memory, the translation in the memory is directly used for translation. Under normal circumstances, it is difficult for the translated input sentence to have a completely matching translation. At this time, the system will calculate the similarity and return the Top N translations in the translation memory that are closest to the input sentence to the user for reference. At the same time, according to the similarity, it is sorted for easy selection by the user. Another special case is that the sentence in the translation memory does not match the sentence to be translated at all; that is, the sentence entered by the user is a completely new sentence (this happens frequently for translation memory with a small corpus). At this time, the user needs to perform manual translation, and the system automatically recovers and stores the translation results as translated sentence pairs. The technical composition involved in the computer-aided translation system is shown in Figure 3. 6 Wireless Communications and Mobile Computing Through the above process, the process of extracting the optimal context in the translation process is completed. System fusion technology has been successfully applied in many fields and has brought considerable performance improvement on the basis of the fused system. In a certain task, when multiple system output results are obtained, the system uses the information in the process of the task or only uses the output results to process the multiple system outputs to obtain a final fusion result to optimize the task output quality. In machine translation, the significance of system integration is that different methods are used to build models and different translation techniques will be used to produce different translation results. Different models and translation techniques have different implementations, with their own strengths and weaknesses. System fusion technology is aimed at multiple systems and integrates the translations of these systems, in line with the purpose of promoting strengths and avoiding weaknesses, to achieve the effect of producing better translations. System fusion technology has been successfully applied to many fields of natural lan-guage processing, and more and more people now apply system fusion technology to statistical English translation technology and have achieved good results. The principle of the English translation system fusion is shown in Figure 4.
In the interactive case, the input of the system is not only the source language sentence but also the translation prefix confirmed by the user. Search decoding in an interactive environment becomes a restricted decoding process; that is, paths that do not satisfy the restriction are not considered. Figure 5 shows a framework diagram of a phrase-based interactive machine translation system.

Experiment Preparation for Interactive English Translation System
The English online-assisted translation system based on human-computer interaction has adopted the B/S threelayer structure. Through the two different processing methods of the front end and the back end, through the modular design, the functional modules of the whole system Computer-aided translation system   Figure 6. Table 1 lists the hardware configuration of the operating environment and the necessary software versions for the front and back ends.
The corpus used in the experiment is a Chinese-English parallel corpus, which contains a total of 66,530 parallel sentence pairs. 5000 sentences from the total corpus were manually extracted as the development set. In addition, 5000 sentences were randomly selected as the test set, and the rest were used as the training set. For the training, development, and test sets, some basic information statistics are made, and the statistical results are shown in Table 2.
A translation phrase pair is the basic unit of a phrasebased statistical machine translation system. An abstract representation needs to be learned for each phrase pair as the representation of the neural network leaf nodes. A simple way is to direct and supervise the learning of the representation of each phrase pair through bilingual data based on the above training process. Such learning is similar to the learning of lexical representations. However, since the amount of bilingual data is far less than the amount of   Table 3 presents the relationship between training data and model parameters. On the dataset, the amount of data used to train the monolingual vocabulary representation is 1G, while the monolingual vocabulary size is only 500 K. For the word pairs composed of the source language and the target language, there are a total of (500 K) 2 , and the training data is limited to a small amount of bilingual data.

Data Deconstruction of the Design and Implementation of the Interactive English Translation System
This paper has selected the corpus in the test set as the analysis object of this paper and compared and described the corpus of the translation and the official reference translation word by word. Figure 7 shows the statistical results of the frequency of error types in the machine translation-to-English system. This paper has divided many types of errors. Due to the mutual influence of certain types of errors, the problem of translation quality is caused by the synthesis of multiple errors. Therefore, it is neither realistic nor scientific to propose correction strategies for these types of errors alone.
The translation platform performed poorly under certain error types, with discourse errors reaching 120 times. The types with typical significance and analytical value were selected for in-depth description and diagnosis. Figure 8 shows the statistical results of ontology errors in English translation errors. As can be seen from Figure 8, from the frequency statistics of error types, the frequency of ontology errors accounts for a low proportion of the total error frequency, but the original text recognition and sentence segmentation errors are very concentrated, much higher than other error types, with a frequency of 20 and 23. From the performance of the machine translation platform, the English translation performance is relatively good, and the performance in the original text recognition and sentence segmentation errors is good. The problem of Chinese recognition and sentence segmentation is still the focus of machine translation researchers. In terms of recognition and sentence segmentation, the main mistakes of machine translation are as follows. (1) It is unable to correctly identify Chinese sentence structures, especially parallel structures, biased structures, and subject-predicate structures. (2) It cannot handle complex sentence structures well. Therefore, it can only be stacked literally and cannot convey any effective meaning in English. (3) Improper sentence segmentation will cause the machine translation to be too long, causing readers to have difficulty in reading. Figure 9 is a frequency map of semantic errors in segfaults. As can be seen from Figure 9, except for the frequent   Testing the accuracy of the system designed in this paper to implement the English translation system, the results are shown in Figure 10.
The analysis of Figure 10 shows that the precision and recall rate of machine English translation using this system are relatively high. When the number of iterations is small, the precision rate is increased by more than 20% compared with the existing English translation system. And after 90 iterations, it can reach 100% with better performance.

Conclusions
The IoT is a significant opportunity for the development and change of information technology. Along with the fast growth of information science and technology and the acceleration of globalization, the problem of language barriers in language communication among countries is becoming more and more prominent, and the need for effective translation methods is becoming more and more urgent. In some way, the evolution of interactive English language translating technique has combined with English translation technique. The level of machine translation technology largely determines the practicality of interactive translation technology.
Through several years of progress, impressive results have been achieved in both fields. However, it should be noted that the operational use of machine translation is far from what one would expect, while interaction machine translation techniques are undergoing rapid development with the aid of interpreters. This can be said to be a transition in the recognition of the career development of translation

11
Wireless Communications and Mobile Computing and can also be seen as an application of the innovative results of the stage development of machine translation. In this paper, a frame for an English translation system was devised. In the interactive translator system designed in this paper, the English translation system is performing well. It is hoped that the research in this article will lead to accurate translations in the English-Chinese translation.

Data Availability
No data were used to support this study.


Enriching a Fashion Knowledge Graph from Product Textual Descriptions

Knowledge Graphs offer a very useful and powerful structure for representing information, consequently, they have been adopted as the backbone for many applications in e-commerce scenarios. In this paper, we describe an application of existing techniques for enriching thelarge-scale Fashion Knowledge Graph (FKG) that we build at Farfetch. In particular, we apply techniques for named entity recognition (NER) and entity linking (EL) in order to extract and link rich metadata from product textual descriptions to entities in the FKG. Having a complete and enriched FKG as an e-commerce backbone can have a highly valuable impact on downstream applications such as search and recommendations. However, enriching a Knowledge Graph in the fashion domain has its own challenges. Data representation is different from a more generic KG, like Wikidata and Yago, as entities (e.g. product attributes) are too specific to the domain, and long textual descriptions are not readily available. Data itself is also scarce, as labelling datasets to train supervised models is a very laborious task. Even more, fashion products display a high variability and require an intricate ontology of attributes to link to. We use a transfer learning based approach to train an NER module on a small amount of manually labeled data, followed by an EL module that links the previously identified named entities to the appropriate entities within the FKG. Experiments using a pre-trained model show that it is possible to achieve 89.75% accuracy in NER even with a small manually labeled dataset. Moreover, the EL module, despite relying on simple rule-based or ML models (due to lack of training data), is able to link relevant attributes to products, thus automatically enriching the FKG.

Introduction
Knowledge Graphs (KGs) have proven to be essential in various domains including e-commerce as they power many important business applications such as search and recommendation. It is no exception for Farfetch 1 , a global ecommerce platform for luxury fashion, where we have recently embarked on Fig. 1. Example of a product that has no attributes specified but several attributes can be extracted from the description such as material, fastening type, toe shape, and sole type.
the mission of building our own Fashion Knowledge Graph (FKG). Currently, the FKG contains entities like products, their attributes, such as color or material, categories, product lines, brands, aesthetics, etc., that in total amount to a few billion triples.
To keep the FKG up to date we have built scalable data pipelines that regularly update the graph with thousands of products and metadata so it always contains fresh and relevant data. The source data largely comes from structured data, e.g., internal data warehouses that are populated from web forms for product information that are filled in by the back office or sellers. As the latter process is manual and restricted to particular attribute types, it is often the case that products miss a lot of important information which leads to an incomplete KG, and thus to under-performing downstream applications. Therefore, as part of our KG construction pipeline, we make use of Machine Learning models that extract product information from unstructured data such as images and text. For instance, for the product depicted in Figure 1 the seller did not specify any attribute information, but these can be extracted either from the image, such as color information, or from the description, such as material, fastening type, toe type, and sole type. To extract such information from product descriptions, as part of our FKG construction pipeline we have built Named Entity Recognition (NER) and Entity Linking (EL) modules. The NER module takes a product description as input, and tags sequences of strings, referred to as mentions, with a possible attribute type. The EL module then links these tagged sequences to the existing attribute values that are stored in the FKG. For example, for the product in Figure 1, the NER module would tag the string "PVC" as material, then the EL module would link this mention to :pvc, an entity in the FKG of type :Material.
When building a KG for the e-commerce domain, we need to consider a few challenges. First, there is the sheer complexity of the domain, with thousands to millions of products types, organized into different taxonomy trees. In addition, there is a big variety across product types regarding the attributes. Not only the attributes are different when considering different types of products, but the attribute values vocabularies also vary across the different attributes. Together, these challenges result in a lack of labeled data for the full taxonomy of products in our catalog.
In this paper we demonstrate a simple approach for extracting mentions from product textual descriptions, linking them to existing attribute values stored in the FKG, and consequently enriching the products with new attribute metadata. As main contributions of this work, we: demonstrate how we leveraged transfer learning to train an NER model, based on BERT [7] Transformer model, for the fashion domain with very limited labeled data; outline how simple rule-based models or traditional ML models using simple features can be used to build an EL system; report some enrichment statistics after applying the NER and EL components, end to end, to the textual descriptions of a set of products in our catalogue.
The rest of the paper is organized as follows. Section 2 gives an overview of some of the relevant related works. In section 3, we describe the adopted methods, including a formalization of the problem followed by a detailed explanation of each method. Section 4 lists all the experiments done, including details of the small in-house labeled dataset used to train the NER model. Lastly, section 5 reports the experimental results and analysis, followed by the conclusions drawn in section 6.

Background and Related Work
As previously introduced, our approach can be interpreted as combining two main steps: first a Named Entity Recognition (NER) task and, second, an Entity Linking task (EL). In the literature, NER can be either seen as a component of EL to detect the mentions in the text, or can be also handled independently from the EL system. At Farfetch, we had the manual resources available to annotate a small NER dataset and so we decided to have the NER as a separate independent component. In accordance, we will review the relevant state-of-the-art methods for NER and EL separately.

Named Entity Recognition
The Named Entity Recognition (NER) task has been widely used in the information extraction domain to identify entities, from unstructured text, and to classify them into pre-defined categories (like names of persons, organizations, or attributes). [23] surveys long-established NER approaches, prior to transformers.
First approaches were commonly based on feature engineering and other supervised or semi-supervised learning algorithms, including Support Vector Machines (SVMs), Conditional Random Fields (CRFs) or Hidden Markov Models (HMMs) [23]. With the "deep learning" revolution, a leap in benchmark performance was achieved by both word-level and/or character-level architectures based on applying a bidirectional-LSTM layer to word embeddings (namely word2vec or glove) and then obtaining a final prediction from a CRF layer [9,11]. LSTMs are very suitable for capturing the semantics and context of the entities as they are able to handle sparse context and have a sequential nature. On the other hand, LSTMs have been combined with CRFs to enforce tagging consistency and to extract cohesive and meaningful entities classifications. Attention mechanisms have also been applied (between the LSTM and CRF layers) in order to increase NER model's interpretability, i.e. to help explain the model's decisions by highlighting the importance of key concepts relative to their neighborhood context, as in [25]. Particularly, [25,22] are framed in the e-commerce domain and also use product textual information, such as title, descriptions, and bullet points. These works tackle the problem of Attribute Value Extraction (getting all attribute values for a set of products) and model it as a sequence tagging task, thus can be seen as a special case of NER. Marcelino et al., (2018) [13] proposed a hierarchical deep learning natural language parser for fashion. Their approach not only recognized fashion-domain entities but also exposed syntactic and morphologic insights.
Transformers [21] have indelibly changed the NER landscape. Current stateof-the-art NER methods are dominated by the Transformer architecture [21] which handles sequential input data, like LSTMs, but solely relies on a selfattention mechanism. Such architecture brought significant performance and efficiency improvements and Transformers blocks have been extensively used for learning language representations applicable to many NLP downstream tasks. Popular Transformer models pre-trained on very large corpora, that have been fine-tuned to tackle the NER task, among others, include BERT [7], that introduces a "masked language model" (MLM) pre-training objective, XLM-RoBERTa [6], which is also based on MLM but outperforms multi-lingual BERT on NER benchmarks. Employing a pre-trained (self-supervised) BERT-like model leverages the power of transfer learning in NLP, helping to solve the NER task for specific domains where data is scarce, as we only need to fine-tune it on the specific domain data.

Entity Linking
Entity Linking is the process of linking entity mentions appearing in unstructured text with their corresponding entities in a knowledge base (KB). Although Entity Linking can be applied to any kind of KB, in this review we will focus on methods that leverage the structured knowledge that we can find in knowledge graphs.
Apart from NER, an entity linking system has other three main components: candidate generation, which is responsible for retrieving candidate entities for each mention from a KB; candidate ranking/disambiguation, which is responsible for scoring and/or ranking the candidate entities and choosing the one (or ones) to link; and, finally, unlinkable entities prediction (also called NIL clustering) that deals with the mentions that were not linked to any entity.
Before focusing on specific relevant methods found in the literature for each of these components, and for the sake of completeness, we point the reader towards [18,19,1], where the first two surveys comprehensively review EL methods based on Deep Learning, while the latter presents an overview of recent advances in tasks for lifting Natural Language texts to KGs.
Approaches to candidate entity generation have been primarily based on string matching techniques, both for named dictionary based techniques and for surface form expansion [20]. In such techniques, string matching is performed between the mention string (or the surface form of the entity mention) and the name of the entity existing in the KB, usually using a named dictionary. Moreover, some entity linking systems try to leverage the whole Web information to identify candidate entities via Web search engines [20].
The majority of proposed methods in the literature have either focused only on the candidate ranking/disambiguation component or, more recently and with the advent of deep neural networks, are end-to-end joint approaches. The latter approaches deal directly with an input text and aim to extract all candidate mentions and link them to their corresponding entities in a KB.
Within disambiguation only approaches, Hoffart at al. [8] unifies prior approaches into a comprehensive framework that combines three measures: popularity prior, similarity, and coherence. In addition, it also introduces new measures for defining mention-entity similarity, and a new algorithm for computing dense sub-graphs in a mention-entity graph, which produces high quality mention-entity mappings. However, the main contribution of [8] is one of the best known benchmarks in the entity disambiguation task: the AIDA CoNLL-YAGO -a new dataset based on CoNLL 2003, in which they manually annotated all proper nouns with corresponding entities in YAGO2.
Recently, transformers have been widespreadly used in disambiguation tasks, leading to performance leaps on multiple benchmarks. Specifically, Yamada et al. [24] introduce a confidence-order model that achieves top results in several benchmarks, by making use of the recent pre-trained BERT model. In this work, the masked language model task is tackled as a masked entity prediction task. Considering the context provided by KGs, Mulang et al. [15] demonstrate that pre-trained transformer-based models, although powerful, are limited to capturing context available purely on the texts concerning the original training corpus. They observe that adding an extra, task-specific, KG context improves the performance of Candidate Entity Disambiguation methods, leading to a new best performance for AIDA-CoNLL dataset.
As for end-to-end approaches, relevant NN-based end-to-end linking models include: Ment-norm [12], Kolitsas et al. [10], NCEL [5], and Martins et al. [14]. Ment-norm [12] achieved the best end-to-end results followed by Kolitsas et al. [10]. Kolitsas et al. [10] use a bi-LSTM based model for mention detection and computes the similarity between the entity mention embedding and a set of predefined entity candidates. The authors demonstrate that engineered features are almost unnecessary when using end-to-end approaches. Their model reaches SOTA results on the AIDA/CoNLL dataset (comparing with the other end-toend methods) and, when combined with Stanford NER, it generalizes well to other datasets with different characteristics. Ment-norm [12] is an end-to-end system for NEL that considers relations as latent. Representation learning was used to learn relation embeddings, eliminating the need for extensive feature engineering. Martins et al. (2019) [14] also explore joint learning of Named Entity Recognition (NER) and EL showing that the two tasks benefit from joint training. More recently, Cao et al. [3] propose GENRE, a novel paradigm to address entity retrieval: it generates entity names autoregressively (leveraging pre-trained autoregressive language model BART and tackling EL as sequenceto-sequence problem). m-GENRE [4] extends GENRE [3] to multi-language. In [2], Cao et al., make an improvement over the previous study (GENRE [3]) that boosts performance as it parallelizes autoregressive linking across all potential mentions and relies on a shallow and efficient decoder. Moreover, they add a new discriminative component that optimizes generators ranking. Finally, Ravi et al. [16] propose CHOLAN, a modular approach to target end-to-end entity linking. CHOLAN consists of a pipeline of two transformer-based models integrated sequentially to accomplish the EL task. The first transformer model performs Named Entity Recognition in a given text and the second performs Named Entity Disambiguation (using the Wikipedia pages as entities external context).
Comparing disambiguation-only with end-to-end methods, taking into account benchmark results reported in the previously mentioned works, we observe that disambiguation-only methods usually outperform end-to-end methods in the entity disambiguation task. However, the recent GENRE [3], has competitive results for entity disambiguation and achieved SOTA in the entity linking task. Nevertheless, end-to-end models are usually based on deep networks which need a huge amount of training data to generate robust models that perform well. In fact, [10] has shown that the accuracy of end-to-end models drops dramatically when trained on or applied to small datasets. This becomes a serious limitation especially when we're performing EL in specific (small) domains, as is the case of in work.

Method
Recently at Farfetch we embarked on the journey of building our own Fashion Knowledge Graph (FKG). The purpose of the FKG is to model and structure our knowledge about fashion and unify information about the main entities such as products, brands, aesthetics, etc. Such unified information can be effectively used in various applications such as search and recommendations.
The FKG is modelled as a set of RDF 2 triples, i.e., tuples of the form (s, p, o), where s and p, called the subject and the property, are an IRI, and o, called the object, is an IRI, blank node, or a literal. In the FKG, entities can be instances of different classes, such as :Product or :Brand, and be connected via properties such as :hasBrand and :hasAesthetic. Since in this paper we are interested in enriching the FKG with the product attributes extracted from the textual descriptions, we therefore enrich only a single property :hasAttribute that connects products to its attributes in the FKG. Rest of the data for FKG, on the other hand, comes from other internal or external data sources and enrichers.
In what follows, we outline the methods involved in extracting the attribute mentions from the textual product descriptions and linking them to the attribute in the FKG. Attribute mentions are the sequence of strings extracted from the textual product descriptions that describe a possible attribute value. First, we define and formalize our task. Next, we detail our method for detecting attribute mentions in unstructured text. Finally, we explain how these mentions can be linked to the attribute values presented in the FKG.

Task Definition
Given a textual product description, our objective is to extract relevant product attributes. The relevant attributes are the attributes presented in our Fashion Knowledge Graph.
We formulate this task as a two step process. The first step is a Named Entity Recognition (NER) component, which detects attribute mentions in the unstructured text. In addition, NER also detects the type of attribute associated with each mention. Formally, consider a product P and its textual description T P = (t P 1 , t P 2 , ..., t P n ) where t P i refers to the i th token of the product description. The named entity recognition task requires to detect a set M P = {m P 1 , m P 2 , ..., m P k }, k ≤ n, such that every m P j is a pair of mention and mention type. In the following we often use the notions of mentions and mention pairs interchangeably. For the example in Figure 1, the NER detects the following attribute mention and mention type pairs: ("PVC", material), ("lace-up front fastening", closure type), ("round toe", toe shape), and ("ridged rubber sole", sole type).
The second step is an Entity Linking (EL) component, in which the detected attribute mention pairs are linked to a set of entities from the FKG. Formally, consider the previous detected attribute mentions M P . In addition, we are given a set of attribute entity and type pairs E = {e 1 , e 2 , ..., e l } extracted from the FKG. The entity linking task is the task of linking each mention m P j ∈ M P to a unique entity e P i ∈ E ∪ {NIL}, where NIL is a special null symbol to denote that no entity is linked. As an example, consider the following set of entity and entity type pairs, corresponding to specific product attributes, retrieved from -("round toe", toe shape) → (:round, :Toe Shape) -("ridged rubber sole", sole type) → NIL

Named Entity Recognition
Due to the lack of labeled NER data within the Fashion domain, we make use of a pre-trained Transformer language model that has already been trained on a large amount of unstructured text. We take the weights from the pre-trained BERTbase-cased model [7] and fine-tune it on top of a small set of manually labeled NER dataset. We used an in-house built user-interface tool to manually label 900 descriptions of products from dresses category across 11 classes. We used the standard BIO 3 (short for beginning, inside, outside) notation for labelling the named entities, a common tagging format for tagging tokens in a chunking task in computational linguistics. The B-tag is used to indicate that the token is the beginning of a named entity phrase, while the I-tag is used for tokens inside the phrase, and the O-tag is used for tokens that do not belong to any named entity chunk. A sample product description with labeled named entities is shown in Figure 2.

Entity Linking
As explained in Section 2, traditional entity linking (EL) systems are composed of three main components: candidate entity generation, candidate entity disambiguation and NIL clustering. In our solution, we only focus on the first 2 components. Figure 3 describes the full end-to-end system.
Candidate Entity Generation can be reformulated as an Information Retrieval (IR) problem: given a query, i.e. an attribute mention, we want to retrieve a set of results relevant to that query, i.e. a set of candidate entities. Of the variety of IR techniques, the most widely used are based on the BM25 [17] algorithm. BM25, where BM is an abbreviation of best matching, is a non-linear retrieval function that combines three key document attributes: term frequency, document frequency, and document length. There are a lot of available search Fig. 3. Full end to end system. Given a product textual description, the NER component detects attribute mentions and their corresponding types in the text. Next, for each attribute mention, the candidate entity generator retrieves a set of candidate attribute entities. Lastly, the candidate entity disambiguation predicts a score for each candidate, and decides which candidates should be linked or not. In the example described in the diagram, one of the attribute mentions detected is "white", which is tagged as color. Given this mention, the following candidate entities are retrieved: (:white, :Color), (:off-white, :Color), and (:white gold, :Material). Since the entity (:white, :Color) is the one with higher score, and because the score is larger than a defined threshold, the entity is linked.
engines that allow us to search for content based on inverted indexes, using the BM25 algorithm as a ranking function.
The attribute values in the FKG are represented as entities with labels as string literals. In order to be able to search over the attribute values in the FKG, we first create an inverted index for all their labels, and then use that index to perform full-text search using the BM25 function to rank the results.
Candidate Entity Disambiguation is formulated as a binary classification problem. Given a pair of an attribute mention and a candidate attribute entity, we predict if the candidate should be linked or not. We highlight, at this point, that we lack the training data needed to train more complex solutions based on Neural Networks. Therefore, we test two types of models: rule-based models and traditional ML models.
The rule-based models are based on domain-specific heuristics. First, we build a consistent text-preprocessing pipeline that enforces lower-casing, text normalization (including accents normalization and special characters removal), and stemming. Next, we create a type mapper which only allows certain types of entities to be linked with certain types of mentions. Finally, we build two different rule-based models. One of them performs an exact-match between the mention and the candidate entity tokens. The other verifies if the candidate entity tokens are contained within the mention tokens, a matching that we name as sub-match. We do this because we want to link to a broader entity, but never to a narrower entity in order to guarantee the validity of what is linked. Table 1 exemplifies some matching cases. The traditional ML models we explore are binary classifiers that predict a binary target for each mention-candidate pair. Since these models need numeric features as input, we build a set of features for each mention-candidate pair that are used by the binary classifier to make its prediction. These features can be split into two main types: string similarity features: as the name suggests, these features use string similarity measures. In particular, we use jaro-winkler 4 to generate the similarity scores between the mention and the candidate entity tokens. semantic similarity features: these features use vector representations (embeddings) for the mention and candidate entity, and then compute the cosine similarity between these representations. In order to generate specific embeddings for the fashion domain, we used a pre-trained BERT-based language model, fine-tuned on 236K textual descriptions from our product catalog, using Masked Language Modeling as in [7].
In order to keep the EL independent of the attribute types, not only do we generate the previous features using the mention and entity names, but we also use their types. This is in contrast with the rule-based models, in which we define a type mapper heuristic based on our domain expertise.
During inference, instead of predicting the binary label directly, we compute a linking score (i.e. the score associated with the positive target) and then rank all the candidate entities, for the same mention, by the linking score. Then, if the score of the top-rank candidate is larger than a specific threshold (that can be tuned during training), the attribute entity is linked.

Experiments
In this section, we outline the experiments conducted for the different components. In the first experiment, we fine-tune a BERT model in an NER task. In the second experiment, we train a few binary classifier models and compare them to our rule-based models, for the candidate entity disambiguation task. Both experiments use a small in-house dataset composed of textual product descriptions from the fashion domain.

Named Entity Recognition
As the world of fashion is a very complex domain with lots of categories and sub-categories arranged in a hierarchical manner, we focused only on Dresses category for our experiments. We fine-tuned a pre-trained Transformer BERTbase-cased model [7] for NER using 900 manually labeled product descriptions across 11 different tags, viz. shape, hem, color, collar, material, fastening, pockets, neckline, sleeve_type, pattern, and detail. The pre-trained model was trained on Wikipedia 5 and the BookCorpus 6 datasets using Masked Language Modeling and Next Sentence Prediction tasks [7]. There were 5702 and 1413 instances of named entities in the training and validation set, respectively, across different tags as shown in Table 2. Here, we used the Hugging Face library 7 to finetune the pre-trained BERT model with the last linear layer on top configured to classify the named entities within our dataset. The model consisted of 12 self-attention heads and 12 hidden layers. Each hidden layer had 768 units and there were 512 maximum positional embeddings. The hidden layer units used gelu activation units with a dropout probability of 0.1. We also used the BERT pre-trained tokenizer consisting of a vocabulary size of 28996 tokens. Hugging Face enabled us to download the model with pre-trained weights.  Train  896 797 759  571  403  699  648  491  352  74  12  Validation 251 203 176  138  74  169  167  116  86  25  8 We fine-tuned the above model on 900 labeled samples, split into 720 for training and 180 for validation. Even though the dataset is small in size, we saw that having access to a pre-trained BERT-base-cased model provides us with the high accuracy necessary for the entity linking step.

Entity Linking
The first component to experiment with was the candidate entity generation. As explained in Section 3.3, we use full-text search over an inverted index of attribute values. In order to increase the recall set of the candidate attribute entities retrieved, we use fuzzy matching. We experimented with fuzziness scores in the range of [0.5, 1.0], using increments of 0.1, and manually tested a small list of attributes that should be retrieved for specific queries. We ended up choosing a fuzziness score of 0.7, which means that we match all terms having 70% matching characters with the query.
Next, we trained and compared different models for the candidate entity disambiguation component. Since we needed labeled data to train (and test) the models, we manually annotated 32 instances, randomly sampled from the dataset used to train the NER model (see Table 2). Each instance contains a product textual description and the corresponding attribute mentions that should be recognized from the text. For each mention, we generate the candidate entities using the candidate entity generation module. We ended up with 2528 samples, where each sample is composed of a product textual description, an attribute mention, and an attribute candidate entity. Lastly, we manually labeled each sample using binary labels, with the positive samples representing the candidate attribute entities that should be linked, yielding 181 (7.2%) positive samples and 2347 negative samples. The final dataset contains 132 unique attribute mentions, covering the 11 unique types described in Table 2, and 522 unique attribute entities covering 33 unique types of attributes (the top 10 attribute types with more samples are represented in Table 3). We used a train/test split ratio of 9:1. Regarding the candidate entity disambiguation models, we used the two rulebased models described in Section 3.3 as baselines, and trained 3 different binary classifiers: a logistic regression classifier, a linear support vector classifier (SVC), and a random forest classifier. As metrics, we used the standard metrics for binary classification: precision, recall and f1-score. We do not report the accuracy score due to the high unbalanced nature of the dataset.

Results
In this section, we report the accuracy numbers for the NER model and the results from the classification model for the candidate entity disambiguation model. In addition, we also report a few enrichment statistics using the full system end to end.

Named Entity Recognition
To evaluate the performance of the fine-tuned BERT-base-cased model, we used the seqeval 8 framework in Python. The fine-tuned BERT model achieved an accuracy of 89.75% on the validation set. Due to the small size of the dataset, we had to stop the training of the model early enough to prevent overfitting. We also visualized the weights of the self-attention layer showing the importance between the tokens in the input text. For example, as shown in Figure  4, the token red has a high correlation with tokens including fashionable and color. This way, we believe that the model learned an inner representation of the fashion domain language that can then be used to extract features useful for downstream tasks.

Entity Linking
We report the binary classification results on the test set, for the candidate entity disambiguation task, in Table 4. As expected, the exact-match rule-based model has a perfect precision. Although there may be a few ambiguous cases in which there is an exact match with multiple attributes, we are constraining the linking based on the mention and entity types using a type mapper heuristic, and therefore disambiguating such cases. For example, consider the attribute mention "long", labeled with the type sleeve_type by NER (recall Table 2). Two exact-matched candidate entities are (:long, :Length) and (:long, :Sleeve Length). Due to the type mapper heuristic, only the (:long, :Sleeve Length) attribute entity is linked. The high precision of the exact-match rule-based model comes with the trade-off of a low recall (only 56% of the entities are linked).
In contrast, due to its less stringent matching mechanism, the sub-match rule-based model has a higher recall (0.72), while maintaining a decent precision (0.783), and thereby achieving an f1-score of 0.75. Regarding the traditional ML binary classifiers, during training, we estimated the score thresholds for each model in order to keep a minimum precision of 0.9, using cross-validation experiments. The results reported in Table 4 correspond to the results on the test set, using the score threshold estimated during training. Although the logistic regression has the highest precision (0.94), both the support vector classifier and the random forest models keep the minimum precision requirement of 0.9. The random forest classifier is the model that yields the best overall results, with the highest f1-score (0.85), and therefore it is the model implemented in the candidate entity disambiguation component.

End to End System
The previous experiments and results describe the performance of both the NER and EL components individually. In order to estimate how the end to end system is performing, we applied the full system to a set of 650 product textual descriptions, extracted from the dresses category. On average, for each product, we detect 7.79 attribute mentions, we generate 12.8 candidate entities for each mention, and end-up linking 5.68 attribute entities. From all the entities linked, 52% are exact matches.
In addition, there are an average of 2.63 attribute mentions that are not linked per product. Table 5 reports the top 2 mentions without any links, that are detected in a large number of products. The "straight hem" mention represents a limitation of EL, which could easily be solved by improving the text-preprocessing, since we can find the attribute entity (:straight hemline, :Hemline Style) among the candidates. However, regarding the "fitted waist" mention, there are no relevant candidates. Since this mention occurs with high frequency in the products textual descriptions, it is a good indicator that we may need to create a corresponding new attribute entity in the FKG. Although the previous statistics demonstrate that we are enriching the products with an average of 5.68 attributes, and the EL experimental results report a linking precision of 0.9, we are still not completely certain about the correctness of the full enrichment, due to error propagation from the NER component to the downstream modules. Consider the following product textual description excerpt: "a bright blue dress, featuring a". A possible error would be for the NER to detect ("blue", color), instead of ("bright blue", color). This error would lead to the linking of the attribute (:Blue, :Color), instead of (:Bright Blue, :Color), in the EL component. Although not critical, such error propagation raises the need for improving the robustness of EL.

Conclusion
In this paper, we have described how we applied existing methods to tackle the automatic enrichment of our KG within the fashion domain. We tackled the NER and the EL tasks and, despite the challenges brought by the lack of data and the specificity of the domain, we were able to augment our FKG, which is pivotal for the success of downstream tasks. In particular, we have shown how we leveraged transfer learning approaches to fine-tune an NER model to the fashion domain using a small dataset. In addition, by using traditional ML models with simple features, we also built an EL component using a few labeled samples.
We are also confident that the embeddings obtained by fine-tuning the BERTbased model on the fashion domain capture the context and semantics specific to the domain. Hence, these domain-specific embeddings, which are a by-product of our enriching method, may enable several other downstream tasks.
However, our solution still presents several limitations. One of them is that the errors from the NER model may propagate into the EL step as a result of having a 2-step pipeline. This could be tackled by introducing further robustness matching steps. Having a jointly optimized process could also present advantages worth exploring. Nevertheless, such an approach requires the construction of a new dedicated dataset. So far, because of the efforts required for annotating data, our focus has been only on a subset of attributes for the Dresses category, and we must scale this process to other attribute types and product categories in order to cover a larger set of our catalog. Therefore, we are focusing on defining the highimpact attributes for the business, and developing specific individual models targeting them, since it's crucial to guarantee high-quality results. However, for the remaining attributes, we may experiment with recent attribute value extraction methods, as in [22], that incorporate the attribute types directly as features and thereby are able to scale to thousands of attribute types, while requiring few annotated data points for each type. Furthermore, we feel there is still scope for improvement on entity disambiguation, across multiple levels, namely on text pre-processing, better feature engineering by generating richer features, and increasing model complexity.
Besides the above limitations, and as an additional direction for future work, we feel we can take advantage of the NIL clustering to detect new attributes in order to automatically update the ever-growing and ever-changing taxonomy of FKG.