dask_ml.preprocessing
.Categorizer¶
-
class
dask_ml.preprocessing.
Categorizer
(categories: Optional[dict] = None, columns: pandas.core.indexes.base.Index = None)¶ Transform columns of a DataFrame to categorical dtype.
This is a useful pre-processing step for dummy, one-hot, or categorical encoding.
Parameters: - categories : mapping, optional
A dictionary mapping column name to instances of
pandas.api.types.CategoricalDtype
. Alternatively, a mapping of column name to(categories, ordered)
tuples.- columns : sequence, optional
A sequence of column names to limit the categorization to. This argument is ignored when
categories
is specified.
Attributes: - columns_ : pandas.Index
The columns that were categorized. Useful when
categories
is None, and we detect the categorical and object columns- categories_ : dict
A dictionary mapping column names to dtypes. For pandas>=0.21.0, the values are instances of
pandas.api.types.CategoricalDtype
. For older pandas, the values are tuples of(categories, ordered)
.
Notes
This transformer only applies to
dask.DataFrame
andpandas.DataFrame
. By default, all object-type columns are converted to categoricals. The set of categories will be the values present in the column and the categoricals will be unordered. Passdtypes
to control this behavior.All other columns are included in the transformed output untouched.
For
dask.DataFrame
, any unknown categoricals will become known.Examples
>>> df = pd.DataFrame({"A": [1, 2, 3], "B": ['a', 'a', 'b']}) >>> ce = Categorizer() >>> ce.fit_transform(df).dtypes A int64 B category dtype: object
>>> ce.categories_ {'B': CategoricalDtype(categories=['a', 'b'], ordered=False)}
Using CategoricalDtypes for specifying the categories:
>>> from pandas.api.types import CategoricalDtype >>> ce = Categorizer(categories={"B": CategoricalDtype(['a', 'b', 'c'])}) >>> ce.fit_transform(df).B.dtype CategoricalDtype(categories=['a', 'b', 'c'], ordered=False)
Methods
fit
(X, dask.dataframe.core.DataFrame], y, …)Find the categorical columns. fit_transform
(X[, y])Fit to data, then transform it. get_params
([deep])Get parameters for this estimator. set_params
(**params)Set the parameters of this estimator. transform
(X, dask.dataframe.core.DataFrame], …)Transform the columns in X
according toself.categories_
.-
__init__
(categories: Optional[dict] = None, columns: pandas.core.indexes.base.Index = None)¶ Initialize self. See help(type(self)) for accurate signature.
-
fit
(X: Union[pandas.core.frame.DataFrame, dask.dataframe.core.DataFrame], y: Union[ArrayLike, dask.dataframe.core.Series, pandas.core.series.Series, None] = None) → dask_ml.preprocessing.data.Categorizer¶ Find the categorical columns.
Parameters: - X : pandas.DataFrame or dask.DataFrame
- y : ignored
Returns: - self
-
fit_transform
(X, y=None, **fit_params)¶ Fit to data, then transform it.
Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.
Parameters: - X : {array-like, sparse matrix, dataframe} of shape (n_samples, n_features)
- y : ndarray of shape (n_samples,), default=None
Target values.
- **fit_params : dict
Additional fit parameters.
Returns: - X_new : ndarray array of shape (n_samples, n_features_new)
Transformed array.
-
get_params
(deep=True)¶ Get parameters for this estimator.
Parameters: - deep : bool, default=True
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Returns: - params : mapping of string to any
Parameter names mapped to their values.
-
set_params
(**params)¶ Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form
<component>__<parameter>
so that it’s possible to update each component of a nested object.Parameters: - **params : dict
Estimator parameters.
Returns: - self : object
Estimator instance.
-
transform
(X: Union[pandas.core.frame.DataFrame, dask.dataframe.core.DataFrame], y: Union[ArrayLike, dask.dataframe.core.Series, pandas.core.series.Series, None] = None) → Union[pandas.core.frame.DataFrame, dask.dataframe.core.DataFrame]¶ Transform the columns in
X
according toself.categories_
.Parameters: - X : pandas.DataFrame or dask.DataFrame
- y : ignored
Returns: - X_trn : pandas.DataFrame or dask.DataFrame
Same type as the input. The columns in
self.categories_
will be converted to categorical dtype.