Skip to content

[SDV 2.0] Add high_cardinality flag to ordinal and categorical column metadata #2990

Description

@frances-h

Problem Description

Now that the categorical values are included in the metadata, it would be helpful to be able to indicate columns that have been detected/assigned the sdtype categorical but also have a very high cardinality.

Expected behavior

Add the high_cardinality as an optional flag to the metadata for ordinal and categorical sdtype columns. The flag must be a boolean. If high_cardinality is set to True, then range_values is not allowed to be set for the column.

During autodetection, if a high cardinality ordinal or categorical column is detected (>500 unique values) then the high_cardinality flag should be set to True and range_values should not be autodetected for the column.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions