I think this is a great example of how sustainability really is multi-faceted. LED lighting may be great from an energy consumption standpoint, and not as good with respect to finite material usage.
Energy Use vs Material Supply
I think it's important, however, to apply some weighting to the different factors. I don't believe we're yet at a critical point with respect to rare earth material supply, but I believe we are already at a critical point with respect to climate change (primarily, that we have already passed the 350 ppm CO2 threshold that many climate scientists believe is necessary to preserve the climate that humans have flourished in). So, in my opinion, it would be poor strategy to pass up energy-saving technologies in favor of technologies that save raw materials that we aren't running out of yet.
Light bulbs are unique devices, because they're small, but use lots of energy over their lifetimes. Their relative material footprint isn't that big, compared to their energy footprint. LED lights are expensive to produce, and use much more scarce material than incandescent bulbs, but they last for years (or decades).
LED lights are already the most energy efficient lighting solution available for most residential applications, and there is good reason to think they'll soon pass compact fluorescents by a significant margin.
Short Term Supply Constraints
Here is a report on GE's website about rare earth supply constraints. It's important to note, however, that current price spikes in these materials are largely an economic phenomenon. 95% of these materials are currently supplied by China, because China's policy is heavily influenced by engineers who saw this trend coming. China is also imposing export quotas. There are significant rare earth material reserves in the US (e.g. Nevada) and Australia, but the higher cost of labor here/there hasn't yet convinced those countries' companies or governments to ramp up production. It also takes years to bring new mines online. So, I would expect in the future, as climate problems worsen, and China's economy surpasses that of the US, this supply limitation will ease. Whether it will ease enough to satisfy increased demand requires a crystal ball (also a rare earth material).
Recycling
Another issue sometimes brought up is recylability. It's important not to assume that because a material isn't currently recycled, it's not recyclable. This assumption is made repeatedly with electric car batteries. Until products like LED lights are getting disposed of in large numbers, economic factors limit the opportunities for recycling them. That's not a long-term problem though, if LED lights (and other rare earth applications) catch on.
Long Term
Keep in mind that the theoretical potential for renewable energy is enormous. Just between solar and wind power, there is thousands of times more energy hitting the earth than we need (and even the harvestable potential is much more than we need). We're just not harvesting much of it now, in favor of consuming ten million year old sunlight stored in fossil fuels. But, in the long term future, we may be able to (once again) use technologies that use more energy, if the energy is renewable. At that point, we may decide to reduce, or reinvent LED lighting. But for now, I think the large energy saving potential of LED lighting makes it a good medium term (0 - 50 years) sustainability choice.