Describe the bug
A custom deny-list recognizer that omits deny_list_score gets a different score depending on how it is loaded:
| Path |
deny_list_score |
Deny-list pattern score |
PatternRecognizer(supported_entity=..., deny_list=[...]) |
1.0 |
1.0 |
RecognizerRegistry.add_pattern_recognizer_from_dict({...}) |
1.0 |
1.0 |
RecognizerRegistryProvider(registry_configuration={...}) (YAML / dict registry config) |
0.0 |
0.0 |
Cause: CustomRecognizerConfig.deny_list_score in presidio_analyzer/input_validation/yaml_recognizer_models.py declares default=0.0, and the validated model is dumped with that default present, so the loader passes deny_list_score=0.0 to PatternRecognizer even when the YAML said nothing. PatternRecognizer.__init__ defaults to 1.0.
A deny-list match with score 0.0 is filtered out by any positive score_threshold, so a YAML deny list without an explicit score effectively never fires through the provider path.
To Reproduce
from presidio_analyzer import PatternRecognizer
from presidio_analyzer.recognizer_registry import RecognizerRegistry, RecognizerRegistryProvider
conf = {
"supported_languages": ["en"],
"global_regex_flags": 26,
"recognizers": [
{"name": "Titles", "supported_entity": "TITLE", "deny_list": ["Mr.", "Mrs."]}
],
}
reg = RecognizerRegistryProvider(registry_configuration=conf).create_recognizer_registry()
r = next(x for x in reg.recognizers if x.name == "Titles")
print(r.deny_list_score, [p.score for p in r.patterns]) # 0.0 [0.0]
print(PatternRecognizer(supported_entity="TITLE", deny_list=["Mr."]).deny_list_score) # 1.0
reg2 = RecognizerRegistry()
reg2.add_pattern_recognizer_from_dict(
{"name": "T2", "supported_language": "en", "supported_entity": "TITLE", "deny_list": ["Mr."]}
)
print(reg2.recognizers[-1].deny_list_score) # 1.0
Expected behavior
Omitting deny_list_score means the PatternRecognizer default (1.0) on every path. An explicit value is honored as today.
Proposed fix
Change CustomRecognizerConfig.deny_list_score to Optional[float] = Field(default=None, ge=0.0, le=1.0) and dump recognizer entries with exclude_unset=True, so an omitted key never reaches the constructor.
Behavior change
Deny-list matches from YAML registry configs without an explicit deny_list_score change from 0.0 to 1.0. Anyone relying on the 0.0 default should set deny_list_score explicitly. The change must be called out in the PR description and the docs (docs/analyzer/recognizer_registry_provider.md).
Environment
main at 5e2fcea, presidio-analyzer.
Describe the bug
A custom deny-list recognizer that omits
deny_list_scoregets a different score depending on how it is loaded:deny_list_scorePatternRecognizer(supported_entity=..., deny_list=[...])RecognizerRegistry.add_pattern_recognizer_from_dict({...})RecognizerRegistryProvider(registry_configuration={...})(YAML / dict registry config)Cause:
CustomRecognizerConfig.deny_list_scoreinpresidio_analyzer/input_validation/yaml_recognizer_models.pydeclaresdefault=0.0, and the validated model is dumped with that default present, so the loader passesdeny_list_score=0.0toPatternRecognizereven when the YAML said nothing.PatternRecognizer.__init__defaults to1.0.A deny-list match with score 0.0 is filtered out by any positive
score_threshold, so a YAML deny list without an explicit score effectively never fires through the provider path.To Reproduce
Expected behavior
Omitting
deny_list_scoremeans thePatternRecognizerdefault (1.0) on every path. An explicit value is honored as today.Proposed fix
Change
CustomRecognizerConfig.deny_list_scoretoOptional[float] = Field(default=None, ge=0.0, le=1.0)and dump recognizer entries withexclude_unset=True, so an omitted key never reaches the constructor.Behavior change
Deny-list matches from YAML registry configs without an explicit
deny_list_scorechange from0.0to1.0. Anyone relying on the0.0default should setdeny_list_scoreexplicitly. The change must be called out in the PR description and the docs (docs/analyzer/recognizer_registry_provider.md).Environment
mainat 5e2fcea, presidio-analyzer.