Yes, you are on the right track. You need to measure or estimate how much less electricity is used after your optimization, and multiply that by the (average) emissions caused by generating electricity.
1. Calculating electricity usage reduction
If your calculation is done always on the same server, then you may be able to find out how much energy you saved by using an electricity meter. If you cannot measure (because the server is in a remote data center) then you could make an estimate based on the average server energy consumption and on how much faster your calculation completes. Alternatively you could try and contact the data center and ask what energy consumption data they can share.
If your software is run by many people (clients) from all over the world then this will be much more difficult. You would at least need to know or guess how many people are running your code and how often, and then multiply that by an estimate of the average amount of electricity you reduced. If you have access to client device information you can make a more informed guess by taking different devices into account (desktop computers consume much more electricity than smart phones, but phones are probably slower in completing the calculation)
2. Estimating the carbon intensity of electricity
You need to convert the amount of electricity you saved to CO2 equivalent emissions. For this you need the carbon intensity of the used electricity. Again this could be relatively easy if you have a single server. If that is powered by 100% renewable energy, then you can set your carbon footprint to 0 (this assumes the 'in-use' approach to calculation a footprint, see also this question). If you don't know the source of the electricity you can use the carbon intensity of the average energy mix of the country where the server is located.
If you have many users from all over the world running your code on their devices, you would need to use a world average carbon intensity. If data on where the users are coming from is available, you could refine this by calculating a weighted world average carbon intensity. The latter could be difficult as not all carbon footprints have the same scope, so you would need a list of carbon intensities from many countries calculated in a uniform way.
3. Optionally add emissions from other, low-impact activities
Depending on the level of detail you are interested in, you could also estimate and subtract the emissions caused by writing your optimized code and putting it into production. However if your code is run very often, then the footprint of producing it will be relatively small and in that case you can probably leave it out.
There is also a small carbon footprint involved with storing data in the cloud, so if there are big changes there because of your improvement, that could also be included in the calculation.