tl;dr replacing fossil fuels with clean, sustainable alternatives would save ~17-20% of current total global energy consumption.
I'll start by noting that energy efficiency is just a side perk of getting rid of fossil fuels: the primary goals are to reduce greenhouse gas emissions and other pollutants.
Still, let's look at how much less energy would we waste: that is to say, how much total annual energy produced would reduce, if total demand stayed the same, but sources other than fossil fuels were used to meet that demand. Fossil fuels are very wasteful of energy - most of the energy from fossil fuels burnt in power stations gets wasted through cooling towers; in cars, most of the oil energy also goes into waste heat. Depending on what we used to replace them, we could save a lot of waste energy, as well as reducing pollution.
For cars, the most energy-efficient route is to electrify them.
For electricity, the most energy-efficient routes are wind, PV, hydro, tidal, wave: that's because none of those have fuels, so there is no wasted fuel energy in the process of generation.
For heating, the most energy-efficient route would be wide-scale deployment of heat-pumps. However, as that may not be feasible, I'll give an alternative scenario too, where it's met by electrical resistance heating. This would give about the same figure i a proportion of that was met by solar thermal instead.
I've got 2011 figures to hand easily (IEA & UK DECC Global Calculator), so let's use those.
Summary of savings:
420 GW from cars
2 340 GW from coal- & oil- fired electricity
540 GW from gas-fired electricity
========.
3 300 GW saved for starters
+110 to 570 GW from heating (large uncertainty range - see below for details)
That's relative to a global energy consumption of about 19 000 GW per year, so ~17% without heating, and an additional 0.6-3% from heating. Actual savings could be higher, depending on what we did with cooking, heavy transport (trucks, planes, boats), and industrial processes.
transport
Internal combustion engines for cars tend to be about 20% efficient. Electric cars are more like 80%. So, 60% of all energy use for cars would be saved through electrification. Call it 22 EJ per year total car use, equivalent to 700 GW - so 420 GW savings from cars.
It's much harder to calculate for other transport - trucks, water, air - because we don't know what the best non-fossil substitute is. If it's just burning stuff (biomass, biogas, synthetic hydrocarbons) then the energy efficiency will be about the same as now, so no energy savings.
electricity
Coal and oil plants are really inefficient - about 35% or so. The best ones go higher, but for a world-average, 35% is good enough. So if we replaced all that with non-thermal power (wind, water, pv), we'd save twice as much energy as coal & oil electricity actually delivers. It delivers around 37 EJ per year, let's call that 1170 GW: so converting to wind-water-sun would save about 2300 GW from coal & oil-fired electricity.
Gas is about 50% efficient, so replacing gas-generated electricity with wind-water-sun would save as much energy as gas electricity actually delivers. It delivers about 17 EJ per year, that's 540 GW; so converting to wind-water-sun would save about 540 GW from gas-fired electricity.
(Just to be explicit about this: 100% efficiency is in effect what you get with wind, PV, hydro, as there is no fuel to waste. Primary energy and generated electricity are one and the same thing for these generation sources. That's standard practice as well as best practice. I'm putting aside transmission and distribution losses in the case of both fossil and renewable generation - that is, they're assumed to be the same in each case)
space and water heating
Very tricky to get a handle on the typical efficiency of fossil heating systems: here in Britan, it's probably 80% or so. Elsewhere it could be very different. In theory, heat pumps could do almost all of the work. In practice, there are serious questions about what actually gets delivered - particularly here in the UK for domestic installations. Global heating demand is around 34 EJ per year, so around 1080 GW. Converting that to electrical resistance heating might save 10% (going from 80% efficient to 90%), i.e 110 GW. On the other hand, using heatpumps, it might save 50% (putting half of it on heatpumps with an SPF of 3 - see calculation breakdown below), i.e. 570 GW. So, savings from heating could be somewhere in the range 110 - 570 GW.
Calculation breakdown:
Current demand 1080 GW. Assume this is met at 80% efficiency, so it requires 1350 GW supply (1080 / 0.8).
Scenario: half resistance heating, half heat-pumps. Take half of demand (540 GW), and meet it at 90% efficiency - requires 600 GW (540 / 0.9). Take the other half (540 GW), and meet it at 300% efficiency (a very ambitious global average SPF 3 for heat pumps), requires 180 GW supply (540 / 3). Total supply is now 780 GW (600 + 180), saving 570 GW (1350 - 570).
industrial energy use
For high-grade heat, many industrial processes are already electrifying, and there are many different processes with different waste profiles and different substitution issues, so I'm going to leave those for now, to stop this answer becoming any longer than it already is.