A machine learning engineer is trying to scale a machine learning pipeline by distributing its feature engineering process. Which of the following feature engineering tasks will be the least efficient to distribute?
Select an option, then click Submit answer.
Reference / correct answer:
Imputing missing feature values with the true median
Most accepted answer: D. Imputing missing feature values with the true median
Community votes: D=1
Selected Answer: D D. Calculating the median is computationally expensive in a distributed system because it requires sorting, a global operation involving data shuffling, and node coordination. In contrast, calculating the mean is efficient as it only requires summing and aggregating results across partitions. upvoted 1 times
B. Target encoding involves replacing each category of a categorical variable with a statistic related to the target variable (like the mean of the target for that category). upvoted 2 times
would argue that the answer is b - Target encoding (also known as mean encoding) involves replacing each category in a categorical feature with the mean of the target variable for that category. This process is more complex and challenging to distribute efficiently because it requires calculating and applying the mean target value for each category. upvoted 3 times
