Rain water is generally the safest source of drinking water, as long as you don't live in a chemically polluted environment. To remove heavy metals etc. you'd need active carbon filters or use distillation. But if you live far from big industry, you can assume rain water is fit for drinking. You could boil it to feel safer on top of that.
Ground water, if taken from a deep depth, is filtered by ground layers. It will contain mud, but you just need to let it stay for a while and take water from top. This is how it is done in a well.
Because of bacteriological and parasite hazards, you should always cook water that you draw this way, as long as you are not sure it's safe to drink it raw. In my family village, every household had its own well, but some of them were better to drink. Most of the people were taking water for drinking and cooking from the neighbor who was the luckiest when it came to drinking water.
You can't use water from wells near heavy industry, latrines, pastures, etc. without distillation. Only people who are used to it from childhood can drink such water. I've observed it in Georgia, where children were drinking water from a garden hose. If we drank that water, we'd probably get heavy diarrhea the next day.
In worst case, when you have only contaminated water, you can try to build distillation facility. It requires a lot energy (you can use solar energy). I'm not sure if it is safe with some industrial toxins that evaporate easily, but it's enough for organic toxins such as those from bacteria.