This is a rather niche topic but I'm posting this, because I'd like this info to be out there for anyone who might be experiencing the same issue as me.

I recently had a mermaid sim that would not switch into her fish tail when swimming. After a bit of tinkering, I managed to fix the problem completely on accident and I found out that the problem had to do with me using a custom tail on her. The issue appeared after my sim was invited to a costume party in autumn, at which she dressed up as a chef. From what I can tell, this must have created a second special outfit (the first one being her fish tail), which later must have somehow been made number 1 (possibly because she was somehow reset at some point). Because I was using a custom mermaid tail, the game couldn't tell that the Special outfit 2 was the mermaid outfit she was supposed switch to when swimming. The issue could be partially fixed by changing her scale color via MasterController (I suppose the mirror interaction would work just as well), which would create a third special outfit that used the default EA tail.

However, that still meant I couldn't use my custom mermaid tail, because if I did so via MasterController's Stylist, the outfit would break again. So I set to find a way to delete the Special outfit 1. Given that outfit number 1 in any category cannot be deleted, I couldn't just use MasterController's Stylist to do so. And so, after a bit of messing around with MasterController, I managed to find a rather peculiar solution. The process is as follows:

1. I switched to Special outfit 1 using MasterController.
2. I open MasterController's Stylist while she was wearing Special outfit 1 and deleted Special outfit 2, which contained the fish tail; I was only left with 1 special outfit. At this point, my sim was wearing her chef costume.
3. Using MasterController, I changed her scale color; the sim was wearing the chef's hat while she was in CAS but otherwise she was in her underwear like normal.
4. I made her swim; according to my tests, it doesn't matter if it's a pool or the ocean.
5. When she made that spinning motion to switch to her mermaid outfit, she remained human and I got a GoHere error (in the ocean however I got multiple ones); in a pool the she swam like a mermaid but without a tail (just like the broken mermaids in Isla Paradiso) and in the ocean she got stuck in an endless loop of trying to dive forward but being reset (you will know when you see it, it's very weird).
6. Then I reset the sim using MasterController (I don't know if MC's Reset Sim option is any different from the console command or if resetting the sim is even necessary at all but I'd still do it just to be safe).
7. I changed her scale color again (with Master Controller).
8. I got her to swim again and then there was only 1 special outfit, the fish tail, and finally I was able to customize it!

EDIT: The same thing recently happened with a sim that got turned into a mermaid when he already had 3 special outfits. The fish tail was Special 4 and the first time I used a custom tail it worked fine; the outfit seems to have broken the second time. I followed the exact same fix and it worked, the only difference being that he didn't have any leg scales in CAS and that he was wearing gray underwear rather than black.