I think it depends on what the story is going for.

Colorblind casting in a story like Princess Bride or Stardust, where the entire thing is a completely fantastical escapade that isn't supposed to have any realistic worldbuilding, absolutely. Or even something like Bridgerton, where the period setting is really just elaborate set dressing for a story that isn't supposed to be even remotely grounded in real historical context.

In a context where the worldbuilding of how and where different populations came to be is important to the story, instead of colorblind casting it would be nice to see some acknowledgement that different ethnic backgrounds have some meaning for the people who come from them. For example, the Six of Crows duology has a diverse cast whose backgrounds matter and their existence in the main setting of Ketterdam makes sense with it being a merchant hub. This is a good way of bringing in diversity that doesn't feel thoughtless and tokenistic.

I think it's better to acknowledge that where people come from and how they look actually does matter in the course of their life and can't just be brushed over and ignored in a serious fantasy story that cares about other aspects of worldbuilding.