• That’s my point. They claim to reduce misrepresentation, while at the same time they erase a bunch of correct representations.

      Going back to what I was saying: fine tuning doesn’t increase diversity, it only shifts the biases. Encoding actual diversity would require increasing the model, then making sure it can output every correct representation.

      • It doesn’t necessarily have to shift away from diversity biases. I think with care, you can preserve the biases that matter most. That was just their first shot at it, this seems like something you’d get better at over time.

        • I guess their main shortcoming was the cultural training set. I’m still unconvinced that level of fine tuning is possible without increasing model size, but we’ll see what happens if/when someone curates a much larger set with cultural labeling.

          The labels might also need to be more granular, like “culture:subculture:period”, or something… which is kind of a snakes nest by itself.