Generally, it makes sense to save power, even during the winter.
The difference between space heating and waste heat
The main difference between home heating systems and waste heat from electrical appliances is the way the heat is released, and where it's released. Many types of home heating systems are designed to provide heat in a specific way, such as a fan heater pointing in the general direction of someone feeling cold, a radiant heater pointing towards a family while they watch TV, or a fireplace in the living area.
Heating that is well targeted at the recipients uses less energy than heating that just increases the ambient temperature of the building. This is very much the case in a poorly insulated building where there's not really much point trying to heat the whole building; the heat is lost very quickly anyway.
Insulation, and the type of heating system, makes a big difference
In a well insulated building that runs thermostat-controlled space heating, waste heat from appliances would tend to reduce the load on the space heating system. In that case, one needs to also consider the type of space heating system; if it's efficient or environmentally friendly (like a heat pump, or a system using clean burning renewable fuel), then it's almost certainly better to use the heating system rather than appliances. It is possible though that electricity may be more renewable than the space heating, in which case more detail would be needed.
Also bear in mind that appliances don't last forever. Depending on the appliance, and the profile of its use, the life of the appliance may be longer if switched off each time or if left on.
A good general rule would be to switch off anything not being used that consumes more a negligible amount of power (such as lighting, televisions or computers), but it's not so bad to leave on standby appliances that only use a Watt or two. You can use a power meter to work out which ones don't matter so much. You'd probably find some tvs, computers, stereos etc use lots of power on standby and others use very little, so it's worth checking.