Incorrect NOT IN result due to inferred predicate pushdown over null-aware anti join

[847850277]

Describe the bug

DataFusion returns an incorrect result for a NOT IN subquery when the subquery result contains NULL and another predicate is applied on the left-side join key.

The following query returns id = 2 in DataFusion, but it should return an empty result according to SQL three-valued logic.

CREATE TABLE t(id INT, a INT);
INSERT INTO t VALUES (1, 1), (2, 2);

CREATE TABLE u(k INT);
INSERT INTO u VALUES (NULL);

SELECT id
FROM t
WHERE a != 1
  AND a NOT IN (SELECT k FROM u);

Actual result in DataFusion:

+----+
| id |
+----+
| 2  |
+----+

Expected result:

+----+
| id |
+----+
+----+

To Reproduce

No response

Expected behavior

No response

Additional context

No response

[847850277]

take