'PL61109010140000071219812874'},     'LOCATION': {'<LOCATION>': 'Kilmarnock'},     'PERSON': {'<PERSON>': 'John Doe', '<PERSON_2>': 'Victoria Cherry'},     'PHONE_NUMBER': {'<PHONE_NUMBER>': '999-888-7777'},     'UK_NHS': {'<UK_NHS>': '987-654-3210'},     'US_DRIVER_LICENSE': {'<US_DRIVER_LICENSE>': '999000680'},     'US_SSN': {'<US_SSN>': '602-76-4532'}}In general, the anonymizer works pretty well, but I can observe two things to improve here:Datetime redundancy - we have two different entities recognized as DATE_TIME, but they contain different type of information. The first one is a date (October 19, 2021), the second one is a time (9:30 AM). We can improve this by adding a new recognizer to the anonymizer, which will treat time separately from the date.Polish ID - polish ID has unique pattern, which is not by default part of anonymizer recognizers. The value ABC123456 is not anonymized.The solution is simple: we need to add a new recognizers to the anonymizer. You can read more about it in presidio documentation.Let's add new recognizers:# Define the regex pattern in a Presidio `Pattern` object:from presidio_analyzer import Pattern, PatternRecognizerpolish_id_pattern = Pattern(    name="polish_id_pattern",    regex="[A-Z]{3}\d{6}",    score=1,)time_pattern = Pattern(    name="time_pattern",    regex="(1[0-2]|0?[1-9]):[0-5][0-9] (AM|PM)",    score=1,)# Define the recognizer with one or more patternspolish_id_recognizer = PatternRecognizer(    supported_entity="POLISH_ID", patterns=[polish_id_pattern])time_recognizer = PatternRecognizer(supported_entity="TIME", patterns=[time_pattern])And now, we're adding recognizers to our anonymizer:anonymizer.add_recognizer(polish_id_recognizer)anonymizer.add_recognizer(time_recognizer)Note that our anonymization instance remembers previously detected and anonymized values, including those that were not detected correctly (e.g., "9:30 AM" taken as DATE_TIME). So it's worth removing this value, or resetting the entire mapping
Open In Colab
Open In Colab ->: 'PL61109010140000071219812874'},     'LOCATION': {'<LOCATION>': 'Kilmarnock'},     'PERSON': {'<PERSON>': 'John Doe', '<PERSON_2>': 'Victoria Cherry'},     'PHONE_NUMBER': {'<PHONE_NUMBER>': '999-888-7777'},     'UK_NHS': {'<UK_NHS>': '987-654-3210'},     'US_DRIVER_LICENSE': {'<US_DRIVER_LICENSE>': '999000680'},     'US_SSN': {'<US_SSN>': '602-76-4532'}}In general, the anonymizer works pretty well, but I can observe two things to improve here:Datetime redundancy - we have two different entities recognized as DATE_TIME, but they contain different type of information. The first one is a date (October 19, 2021), the second one is a time (9:30 AM). We can improve this by adding a new recognizer to the anonymizer, which will treat time separately from the date.Polish ID - polish ID has unique pattern, which is not by default part of anonymizer recognizers. The value ABC123456 is not anonymized.The solution is simple: we need to add a new recognizers to the anonymizer. You can read more about it in presidio documentation.Let's add new recognizers:# Define the regex pattern in a Presidio `Pattern` object:from presidio_analyzer import Pattern, PatternRecognizerpolish_id_pattern = Pattern(    name="polish_id_pattern",    regex="[A-Z]{3}\d{6}",    score=1,)time_pattern = Pattern(    name="time_pattern",    regex="(1[0-2]|0?[1-9]):[0-5][0-9] (AM|PM)",    score=1,)# Define the recognizer with one or more patternspolish_id_recognizer = PatternRecognizer(    supported_entity="POLISH_ID", patterns=[polish_id_pattern])time_recognizer = PatternRecognizer(supported_entity="TIME", patterns=[time_pattern])And now, we're adding recognizers to our anonymizer:anonymizer.add_recognizer(polish_id_recognizer)anonymizer.add_recognizer(time_recognizer)Note that our anonymization instance remembers previously detected and anonymized values, including those that were not detected correctly (e.g., "9:30 AM" taken as DATE_TIME). So it's worth removing this value, or resetting the entire mapping