To answer your question about whether or not it's financially worthwhile to swap them out, I looked at two simlarly rated lamps from the same manufacturer:
So there's a 3W difference in power consumption between the two.
If you keep the light on for 4 hours/day every day for a year, it will consume around 4KWh of extra power each year, or around 60 cents worth of power at $0.15/KWh.
The CFL is rated for 8000 hours, so if it's brand new and lasts the entire 8000 hours, it will run for around 5.5 years, costing you an extra $3.30 in electricity. Similarly, if the lamp is already a few years old and has only 2000 hours left it'll cost you less than a dollar of extra electricity.
Unless you can replace the CFL with an LED for less than $3.30, it's probably not going to be financially worthwhile.
Your payback period may be different depending on how often you use the lamp and how much you pay for electricity. I ignored the labor costs of changing bulbs since in most cases it's minimal, but sometimes may be significant -- I have a set of 4 stairway lamps that are mounted high on the wall and I need to bring in the long folding ladder and set it up on the stairs to reach the bulbs. When one of the CFL's burnt out, I replaced all 4 with (hopefully longer lifetime) LED's at the same time since it was easy to replace them while the ladder was set up. It takes me around 30 minutes to get the ladder from the garage, set it up, then put it away again, so if the LED really does last 3 times longer than the CFL, it's economically worthwhile for me to change it out before the end of life of the CFL.
And as others have said, due to the sunk environmental cost in creating the CFLs you already own, it's likely a net environmental loss if you discard them before their useable lifetime is over.