What is the difference between the algorithm value "ignore" and the option "nullMatch" set to false?
The search algorithms can be controlled by setting the
type
field to a specific value. If the type
field is set to ignore
, the algorithm will be configured to ignore the attribute in the search, and any matches will be returned regardless of the value of the attribute. If the type
field is set to any other value, the algorithm will be configured to consider the attribute in the search and only return matches with the specified attribute value.This applies to any attribute that can be used in a search query, such as dates of birth, nationalities, or addresses.
For example, setting the
type
field to same_year
for the datesOfBirth
attribute will only return matches with the same year of birth as the search query, while setting the type
field to ignore
for the datesOfBirth
attribute will return matches regardless of their date of birth.The following examples demonstrate how the
type
field of the dateOfBirthAlgo
algorithm can be used to control the behavior of the search.In the first example, the
dateOfBirthAlgo
is set to same_year
. In this case, no match is found when searching for the individual named 'Abu Husayn' on the USA list because the API only considers matches where the year of birth is the same.{
"queries": [
{
"sourceData": [
{
"names": [
{
"fullName": "Abu Husayn"
}
],
"datesOfBirth": [
{
"date": "1979-04-02"
}
]
}
],
"targetData": [
{
"datasets": [
{
"label": "USA"
}
]
}
],
"threshold": 0.85,
"dateOfBirthAlgo": {
"type": "same_year"
}
}
]
}
As stated above, the search does not return any matches because it does not find any individuals with the same name and year of birth on the USA list.
{
"responseHeader": {
"requestID": "GEN2054093905-1670834053278",
"responded": "2022-12-12T08:34:13.278Z"
}
}
In the second example, the
dateOfBirthAlgo
is set to ignore
, which means that the API will not use the date of birth as a criterion for searching the individual on the USA list. Instead, the API will only consider matches based on the full name.{
"queries": [
{
"sourceData": [
{
"names": [
{
"fullName": "Abu Husayn"
}
],
"datesOfBirth": [
{
"date": "1979-04-02"
}
]
}
],
"targetData": [
{
"datasets": [
{
"label": "USA"
}
]
}
],
"threshold": 0.85,
"dateOfBirthAlgo": {
"type": "ignore"
}
}
]
}
This time, two matches were found.
{
"responseHeader": {
"requestID": "GEN2054093905-1670834263596",
"responded": "2022-12-12T08:37:43.596Z"
},
"results": [
{
"matchType": "st_match",
"score": 1.00,
"matchingNames": [
{
"fullName": "ABU HUSAYN",
"normalized": "abu husayn"
}
],
"sourceData": [
{
"entityType": "unknown",
"names": [
{
"fullName": "Abu Husayn",
"normalized": "abu husayn"
}
],
"cultureCodes": [
"ar"
]
}
],
"targetData": [
{
"dataID": "31971",
"entityType": "individual",
"names": [
{
"fullName": "AL-MADANI, Yusuf",
"normalized": "al madani yusuf"
},
{
"fullName": "AL-MADANI, Yusif",
"normalized": "al madani yusuf"
},
{
"fullName": "AL-MADANI, Youssef",
"normalized": "al madani yusuf"
},
{
"fullName": "AL-MADANI, Yousef Ahssan Ismail",
"normalized": "al madani yusuf ahssan ismael"
},
{
"fullName": "ABU HUSAYN",
"normalized": "abu husayn"
}
],
"sex": "m",
"cultureCodes": [
"ar"
],
"datesOfBirth": [
{
"date": "1977"
}
],
"addresses": [
{
"country": "YE"
}
],
"placesOfBirth": [
{
"city": "Muhatta Directorate",
"country": "YE"
}
],
"active": true,
"keywords": [
"SDGT",
"Specially Designated Nationals (SDN) - Treasury Department"
],
"sources": [
"Specially Designated Nationals (SDN) - Treasury Department",
"https://home.treasury.gov/policy-issues/financial-sanctions/specially-designated-nationals-and-blocked-persons-list-sdn-human-readable-lists",
"https://www.treasury.gov/resource-center/sanctions/SDN-List/Pages/default.aspx"
],
"label": "USA",
"revision": 4968,
"created": "2021-09-30T01:05:02.866",
"updated": "2022-12-12T08:06:01.204",
"lastMatched": "2021-09-30T01:05:02.866",
"lastChecked": "2021-09-30T01:05:02.866"
}
]
},
{
"matchType": "st_match",
"score": 0.98,
"matchingNames": [
{
"fullName": "HUSAYN, Abu",
"normalized": "husayn abu"
}
],
"sourceData": [
{
"entityType": "unknown",
"names": [
{
"fullName": "Abu Husayn",
"normalized": "abu husayn"
}
],
"cultureCodes": [
"ar"
]
}
],
"targetData": [
{
"dataID": "28571",
"entityType": "individual",
"names": [
{
"fullName": "AL-HAMIDAWI, Ahmad",
"normalized": "al hamidawi ahmad"
},
{
"fullName": "AL-HAMIDAWI, Ahmad Muhsin Faraj",
"normalized": "al hamidawi ahmad muhsin faraj"
},
{
"fullName": "AL-SAEDI, Ahmed Kadhim Raheem",
"normalized": "al saedi ahmad kadhim raheem"
},
{
"fullName": "AL-SA'IDI, Ahmad Kazim Rahim",
"normalized": "al saidi ahmad kazim rahim"
},
{
"fullName": "HUSAYN, Abu",
"normalized": "husayn abu"
}
],
"sex": "m",
"cultureCodes": [
"ar"
],
"nationalities": [
{
"country": "IQ"
}
],
"datesOfBirth": [
{
"date": "1974-03-25"
}
],
"addresses": [
{
"city": "Baghdad",
"country": "IQ"
}
],
"placesOfBirth": [
{
"country": "IQ"
}
],
"active": true,
"keywords": [
"SDGT",
"Specially Designated Nationals (SDN) - Treasury Department"
],
"sources": [
"Specially Designated Nationals (SDN) - Treasury Department",
"https://home.treasury.gov/policy-issues/financial-sanctions/specially-designated-nationals-and-blocked-persons-list-sdn-human-readable-lists",
"https://www.treasury.gov/resource-center/sanctions/SDN-List/Pages/default.aspx"
],
"label": "USA",
"revision": 4968,
"created": "2021-09-30T01:05:02.764",
"updated": "2022-12-12T08:06:00.172",
"lastMatched": "2021-09-30T01:05:02.764",
"lastChecked": "2021-09-30T01:05:02.764"
}
]
}
]
}
When analyzing the results, we can notice that the first match comes with a date of birth equal to
1977
while the second one has a date of birth equal to 1974-03-25
. Both years of birth are different from that of the individual sent in the search request. This explains why no match was returned in the first example.The
nullMatch
option is used to specify how search algorithms should handle empty or missing attributes in the watchlists. This option allows the search to be more flexible and consider matches even if some attributes are not provided or are empty. This can be useful in situations where the information in the watchlists is incomplete or not fully known, and the search needs to be able to handle these cases.When
nullMatch
is either not provided or set to true
in a query, the search will consider attributes that are missing or empty as a match, whereas when it is set to false
, these attributes will not be considered a match.For example, if a search query is looking for an Iraqi (IQ) national named 'Abu Husayn' on the USA list and the
nullMatch
option associated with the nationalityAlgo
is set to false
, the search will only return a match if the individual's nationality is explicitly listed as IQ
.{
"queries": [
{
"sourceData": [
{
"names": [
{
"fullName": "Abu Husayn"
}
],
"nationalities": [
{
"country": "IQ"
}
]
}
],
"targetData": [
{
"datasets": [
{
"label": "USA"
}
]
}
],
"threshold": 0.85,
"nationalityAlgo": {
"type": "same_country",
"nullMatch": false
}
}
]
}
In the situation where
nullMatch
is set to false
, only one match is found.{
"responseHeader": {
"requestID": "GEN2054093905-1670836810880",
"responded": "2022-12-12T09:20:10.880Z"
},
"results": [
{
"matchType": "st_match",
"score": 0.98,
"matchingNames": [
{
"fullName": "HUSAYN, Abu",
"normalized": "husayn abu"
}
],
"sourceData": [
{
"entityType": "unknown",
"names": [
{
"fullName": "Abu Husayn",
"normalized": "abu husayn"
}
],
"cultureCodes": [
"ar"
],
"nationalities": [
{
"country": "IQ",
"normalized": "IQ"
}
]
}
],
"targetData": [
{
"dataID": "28571",
"entityType": "individual",
"names": [
{
"fullName": "AL-HAMIDAWI, Ahmad",
"normalized": "al hamidawi ahmad"
},
{
"fullName": "AL-HAMIDAWI, Ahmad Muhsin Faraj",
"normalized": "al hamidawi ahmad muhsin faraj"
},
{
"fullName": "AL-SAEDI, Ahmed Kadhim Raheem",
"normalized": "al saedi ahmad kadhim raheem"
},
{
"fullName": "AL-SA'IDI, Ahmad Kazim Rahim",
"normalized": "al saidi ahmad kazim rahim"
},
{
"fullName": "HUSAYN, Abu",
"normalized": "husayn abu"
}
],
"sex": "m",
"cultureCodes": [
"ar"
],
"nationalities": [
{
"country": "IQ"
}
],
"datesOfBirth": [
{
"date": "1974-03-25"
}
],
"addresses": [
{
"city": "Baghdad",
"country": "IQ"
}
],
"placesOfBirth": [
{
"country": "IQ"
}
],
"active": true,
"keywords": [
"SDGT",
"Specially Designated Nationals (SDN) - Treasury Department"
],
"sources": [
"Specially Designated Nationals (SDN) - Treasury Department",
"https://home.treasury.gov/policy-issues/financial-sanctions/specially-designated-nationals-and-blocked-persons-list-sdn-human-readable-lists",
"https://www.treasury.gov/resource-center/sanctions/SDN-List/Pages/default.aspx"
],
"label": "USA",
"revision": 4969,
"created": "2021-09-30T01:05:02.764",
"updated": "2022-12-12T09:06:38.826",
"lastMatched": "2021-09-30T01:05:02.764",
"lastChecked": "2021-09-30T01:05:02.764"
}
]
}
]
}
Now if
nullMatch
is set to true
or not provided, the same search will also return a match if the individual's nationality is not specified in the list of attributes.{
"queries": [
{
"sourceData": [
{
"names": [
{
"fullName": "Abu Husayn"
}
],
"nationalities": [
{
"country": "IQ"
}
]
}
],
"targetData": [
{
"datasets": [
{
"label": "USA"
}
]
}
],
"threshold": 0.85,
"nationalityAlgo": {
"type": "same_country",
"nullMatch": true
}
}
]
}
In the situation where
nullMatch
is set to true
, the search finds two matches.{
"responseHeader": {
"requestID": "GEN2054093905-1670842976944",
"responded": "2022-12-12T11:02:56.944Z"
},
"results": [
{
"matchType": "st_match",
"score": 1.00,
"matchingNames": [
{
"fullName": "ABU HUSAYN",
"normalized": "abu husayn"
}
],
"sourceData": [
{
"entityType": "unknown",
"names": [
{
"fullName": "Abu Husayn",
"normalized": "abu husayn"
}
],
"cultureCodes": [
"ar"
],
"nationalities": [
{
"country": "IQ",
"normalized": "IQ"
}
]
}
],
"targetData": [
{
"dataID": "31971",
"entityType": "individual",
"names": [
{
"fullName": "AL-MADANI, Yusuf",
"normalized": "al madani yusuf"
},
{
"fullName": "AL-MADANI, Yusif",
"normalized": "al madani yusuf"
},
{
"fullName": "AL-MADANI, Youssef",
"normalized": "al madani yusuf"
},
{
"fullName": "AL-MADANI, Yousef Ahssan Ismail",
"normalized": "al madani yusuf ahssan ismael"
},
{
"fullName": "ABU HUSAYN",
"normalized": "abu husayn"
}
],
"sex": "m",
"cultureCodes": [
"ar"
],
"datesOfBirth": [
{
"date": "1977"
}
],
"addresses": [
{
"country": "YE"
}
],
"placesOfBirth": [
{
"city": "Muhatta Directorate",
"country": "YE"
}
],
"active": true,
"keywords": [
"SDGT",
"Specially Designated Nationals (SDN) - Treasury Department"
],
"sources": [
"Specially Designated Nationals (SDN) - Treasury Department",
"https://home.treasury.gov/policy-issues/financial-sanctions/specially-designated-nationals-and-blocked-persons-list-sdn-human-readable-lists",
"https://www.treasury.gov/resource-center/sanctions/SDN-List/Pages/default.aspx"
],
"label": "USA",
"revision": 4970,
"created": "2021-09-30T01:05:02.866",
"updated": "2022-12-12T10:06:03.700",
"lastMatched": "2021-09-30T01:05:02.866",
"lastChecked": "2021-09-30T01:05:02.866"
}
]
},
{
"matchType": "st_match",
"score": 0.98,
"matchingNames": [
{
"fullName": "HUSAYN, Abu",
"normalized": "husayn abu"
}
],
"sourceData": [
{
"entityType": "unknown",
"names": [
{
"fullName": "Abu Husayn",
"normalized": "abu husayn"
}
],
"cultureCodes": [
"ar"
],
"nationalities": [
{
"country": "IQ",
"normalized": "IQ"
}
]
}
],
"targetData": [
{
"dataID": "28571",
"entityType": "individual",
"names": [
{
"fullName": "AL-HAMIDAWI, Ahmad",
"normalized": "al hamidawi ahmad"
},
{
"fullName": "AL-HAMIDAWI, Ahmad Muhsin Faraj",
"normalized": "al hamidawi ahmad muhsin faraj"
},
{
"fullName": "AL-SAEDI, Ahmed Kadhim Raheem",
"normalized": "al saedi ahmad kadhim raheem"
},
{
"fullName": "AL-SA'IDI, Ahmad Kazim Rahim",
"normalized": "al saidi ahmad kazim rahim"
},
{
"fullName": "HUSAYN, Abu",
"normalized": "husayn abu"
}
],
"sex": "m",
"cultureCodes": [
"ar"
],
"nationalities": [
{
"country": "IQ"
}
],
"datesOfBirth": [
{
"date": "1974-03-25"
}
],
"addresses": [
{
"city": "Baghdad",
"country": "IQ"
}
],
"placesOfBirth": [
{
"country": "IQ"
}
],
"active": true,
"keywords": [
"SDGT",
"Specially Designated Nationals (SDN) - Treasury Department"
],
"sources": [
"Specially Designated Nationals (SDN) - Treasury Department",
"https://home.treasury.gov/policy-issues/financial-sanctions/specially-designated-nationals-and-blocked-persons-list-sdn-human-readable-lists",
"https://www.treasury.gov/resource-center/sanctions/SDN-List/Pages/default.aspx"
],
"label": "USA",
"revision": 4970,
"created": "2021-09-30T01:05:02.764",
"updated": "2022-12-12T10:06:02.691",
"lastMatched": "2021-09-30T01:05:02.764",
"lastChecked": "2021-09-30T01:05:02.764"
}
]
}
]
}
The first match is the same match which is found when
nullMatch
is set to false
, where the individual's nationality is explicitly listed as IQ
.The second match is a new one with no specified nationality.
By default, the API performs searches with the
nullMatch
option set to true
, which means that it will return matches where the attribute associated with a search algorithm is not specified in the watchlists. This is the default behavior of the search API in the absence of the nullMatch
option. To change this behavior, specify the value false
for the nullMatch
option when making a search query.