If yes, then do I have to use a setup?

As far as I can tell you, there is no one way to find out if a mod works besides running the game. If the game crashes, or turns pink, or any other type of malfunction, that's the only way to find out if a mod doesn't work.
It's a hit or miss type of thing, usually resulting in the notorious "50/50" dance you will do to find out what mod is making your game miserable.

There is one program that you should have to find out if a mod is going to interfere with others-HCDU, or the Hack Conflict Detection Utility, available here:
Hack Conflict Detection Utility

Often mods will say what version of the game they are for, and generally speaking you want mods made for (or specifically stated to be working with) your latest EP. Usually mods made for AL will work without issue in M&G or the UC but on some rare occasions this is not the case. Take the time to read everything. Sometimes mods will have multiple versions for various possible EP combinations, and sometimes creators will tell you specific load order that you need in order for mods to work correctly or other mods that are required or recommended to use as well. Definitely run the HCDU to check for potential problems. Add mods slowly, only a few at a time, and check things out in game to make sure things look like they're still working. I find test hoods useful when adding new stuff especially if it's things I want/need to play with for a while when testing and save while playing.

Keep in mind that it's not only mods but also CC that can break things in game; I had a hairstyle that was messing up the scalps of about 40% of my sims. I had to do a full 50/50 to track that down and yank it.

Yeah... I wish there was an easier way than having to look at everything separately, but there mostly isn't. My sims still can't make BFF's autonomously or learn how to study. So I'm in the same boat.

Apart from the HCDU which can show possible clashes that you might need to either remove or change load order on, no. It's called the 50/50 search where you test portions of your download folder. Most times there is nothing wrong with a mod, it's just that two mods don't like working together or you have installed two versions or a version not compatible with your EP set up. Sometimes the mod might need another file to work but it's easy to miss unless you read over everything carefully. It would vary rare that a mod would actually be broken.

If by 'mod' you mean hairs, clothes etc, those are not called mods in sims 2, but Custom Content or CC. I think most of us have had a piece of crashing CC at one time or another, sort by date and remove the last items or again the 50/50 search is the way to resolve it.