dask_ml.preprocessing.Categorizer

class dask_ml.preprocessing.Categorizer(categories=None, columns=None)

Transform columns of a DataFrame to categorical dtype.

This is a useful pre-processing step for dummy, one-hot, or categorical encoding.

Parameters
categoriesmapping, optional

A dictionary mapping column name to instances of pandas.api.types.CategoricalDtype. Alternatively, a mapping of column name to (categories, ordered) tuples.

columnssequence, optional

A sequence of column names to limit the categorization to. This argument is ignored when categories is specified.

Attributes
columns_pandas.Index

The columns that were categorized. Useful when categories is None, and we detect the categorical and object columns

categories_dict

A dictionary mapping column names to dtypes. For pandas>=0.21.0, the values are instances of pandas.api.types.CategoricalDtype. For older pandas, the values are tuples of (categories, ordered).

Notes

This transformer only applies to dask.DataFrame and pandas.DataFrame. By default, all object-type columns are converted to categoricals. The set of categories will be the values present in the column and the categoricals will be unordered. Pass dtypes to control this behavior.

All other columns are included in the transformed output untouched.

For dask.DataFrame, any unknown categoricals will become known.

Examples

>>> df = pd.DataFrame({"A": [1, 2, 3], "B": ['a', 'a', 'b']})
>>> ce = Categorizer()
>>> ce.fit_transform(df).dtypes
A       int64
B    category
dtype: object

>>> ce.categories_
{'B': CategoricalDtype(categories=['a', 'b'], ordered=False)}

Using CategoricalDtypes for specifying the categories:

>>> from pandas.api.types import CategoricalDtype
>>> ce = Categorizer(categories={"B": CategoricalDtype(['a', 'b', 'c'])})
>>> ce.fit_transform(df).B.dtype
CategoricalDtype(categories=['a', 'b', 'c'], ordered=False)

Methods

fit(self, X[, y])

Find the categorical columns.

fit_transform(self, X[, y])

Fit to data, then transform it.

get_params(self[, deep])

Get parameters for this estimator.

set_params(self, \*\*params)

Set the parameters of this estimator.

transform(self, X[, y])

Transform the columns in X according to self.categories_.
__init__(self, categories=None, columns=None)

Initialize self. See help(type(self)) for accurate signature.

fit(self, X, y=None)

Find the categorical columns.

Parameters
Xpandas.DataFrame or dask.DataFrame
yignored
Returns
self
fit_transform(self, X, y=None, **fit_params)

Fit to data, then transform it.

Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.

Parameters
Xnumpy array of shape [n_samples, n_features]

Training set.

ynumpy array of shape [n_samples]

Target values.

Returns
X_newnumpy array of shape [n_samples, n_features_new]

Transformed array.

get_params(self, deep=True)

Get parameters for this estimator.

Parameters
deepboolean, optional

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns
paramsmapping of string to any

Parameter names mapped to their values.

set_params(self, **params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Returns
self
transform(self, X, y=None)

Transform the columns in X according to self.categories_.

Parameters
Xpandas.DataFrame or dask.DataFrame
yignored
Returns
X_trnpandas.DataFrame or dask.DataFrame

Same type as the input. The columns in self.categories_ will be converted to categorical dtype.