I found this interesting research paper that answers some of your questions, at least for US power plants. For people who don't like to read research papers, there's an easier-to-read summary in this Science Daily article.
The research paper discusses carbon dioxide (CO2), nitrogen oxides (NOx) and sulfur dioxide (SO2) emissions from both coal and natural gas power plants by comparing data from continuous emission monitoring systems (CEMS).
The researchers found that natural gas plants that use the "combined cycle" technology release far less CO2, NOx and SO2 than coal plants.
Average CO2 emissions per kilowatt hour:
Coal: 915.0 ± 0.8g
Natural gas: 549.4 ± 1.1g
NG combined cycle: 436.0 ± 1.4g
Sadly the paper does not mention the exact amounts of NOx and SO2, but it does say that
the increased use of natural gas has led to emission reductions of NOx (40%) and SO2 (44%)
Also the paper provides a few graphs so you can get a rough estimate of the amounts of NOx and SO2.

Average emission intensities (in units of g/kWh) of CO2, NOx, and SO2 from U.S. power plants between 1995 and 2012. Error bars give the 1-σ uncertainty in the averages.
Caveats:
- The data is based on US power plant output emissions, so
after filtering (which is not what you asked for)
- Other pollution types such as heavy metals, particulate matter, VOCs and methane leaks were not included in the study.
- Emissions from fuel extraction and transport were also not included.
Methane leaks
Since methane is a very powerful greenhouse gas, leakages from gas production system could potentially negate the positive outcome of natural gas described above.
In 2009 the EPA estimated that the leakage rate is 2.4% of the gross U.S. natural gas production (source). Other researchers think the real amount is much higher, but also say that:
system-wide leakage is unlikely to be large enough to negate climate benefits of coal-to-NG substitution (source).
Surprisingly the same article also mentions that natural gas used as a fuel in cars is most likely not cleaner than diesel.