An organization is developing a feature repository and is electing to one-hot encode all categorical feature variables. A data scientist suggests that the categorical feature variables should not be one-hot encoded within the feature repository. Which of the following explanations justifies this suggestion?
Select an option, then click Submit answer.
Reference / correct answer:
One-hot encoding is a potentially problematic categorical variable strategy for some machine learning algorithms.
Most accepted answer: E. One-hot encoding is a potentially problematic categorical variable strategy for some machine learning algorithms.
Community votes: C=1, E=2
Selected Answer: E A feature repository (feature store) should generally store raw, reusable features rather than transformations that are specific to certain models. One-hot encoding: Works well for some algorithms (e.g., linear regression, logistic regression, neural networks). Can be inefficient or unnecessary for others (e.g., tree-based models such as Random Forest, XGBoost, LightGBM, CatBoost). May dramatically increase feature dimensionality when a categorical variable has many unique values. Since different machine learning applications may use different algorithms, applying one-hot encoding universally in the feature repository reduces flexibility and may negatively affect some models. upvoted 1 times
Selected Answer: E E for sure. Some algorithms do not support or work less efficient with one-hot encoding. upvoted 1 times
Selected Answer: C One-hot encoding introduces a significant computational overhead, particularly when there are many categories in the feature variables. This is because it creates a new binary feature for each category, resulting in a high-dimensional representation. Such transformations are typically more efficient when performed on smaller, task-specific training datasets rather than on a global feature repository that serves multiple applications. Encoding in the feature repository also risks being rigid and unable to adapt to new categories, which further supports your argument. upvoted 2 times
. One-hot encoding is dependent on the target variable’s values which differ for each application. This one doesn't hold up because one-hot encoding isn't dependent on the target variable at all. It only represents categorical features and their individual categories as binary vectors. E. One-hot encoding is a potentially problematic categorical variable strategy for some machine learning algorithms. Certain algorithms, like tree-based models, are less sensitive to the one-hot encoding and might perform better with other encoding techniques. However, the primary issue often boils down to the high dimensionality that one-hot encoding can introduce, which can affect algorithms that don't handle sparse data well. So, you're still left with C as the best justification. upvoted 2 times
It might actually be E. According to these docs, this is the reason why the change was introduced was to allow algorithms that expect continuous features, such as logistic regression to use categorical features upvoted 3 times