I've heard concerns about how long batteries will last (and the cost to replace them)
I wouldn't be worried.
If in an internal combustion engine car your engine gets damaged, most of it will be replaced. For example today cylinder or piston damage means the entire lower end is replaced. That easily costs at least 5000 EUR. There used to be a time in history where cylinders were re-bored to a larger size and pistons were replaced, but not anymore at these labor costs.
If in an internal combustion engine car your transmission gets damaged, it will have to be replaced too. Especially on automatic transmission cars (that are preferred nowadays a lot even in my country Finland that traditionally had strong presence of manual transmission cars) replacing the transmission is very very expensive indeed.
Compare that to electric cars. If part of your battery pack fails, only the part that failed is replaced with a new module. Cost probably below 1000 EUR. If electric motor fails, it's far cheaper to replace than internal combustion engine. If the single-speed transmission fails, it's very cheap to replace too.
About the only problem might be if the entire battery pack has severely reduced capacity. The most durable chemistries are NMC and LFP, and the least durable is NCA. Tesla is using NCA. Guess what? Tesla batteries with 500 000 km generally show very negligible capacity reduction. Based on that, we shouldn't be worried at all about the NMC and LFP packs if their cooling hardware and charging software is good.
Generally battery packs with proper liquid cooling last several times that of a typical internal combustion engine. The only packs that have severely reduced capacity when very young are the Nissan Leaf style packs with air cooling.
Also, car manufacturers take into account battery pack capacity reduction in range estimate. Therefore, if you see a used electric car, you can just look at the remaining range indicator. It knows the battery health and provides information about the true range. So the buyer knows the battery health. A buyer on the other hand doesn't know internal combustion engine or automatic transmission health in an ICE vehicle.
So, the question you shouldn't be asking is about the potential for a used car market for EVs.
You should be asking whether or not there will be a used car market for ICE vehicles.
I believe there won't be. Today there is, but it will fail very soon. The failure of used ICE vehicle market is due to three reasons: (1) high fuel costs, (2) high maintenance costs, (3) it's very hard to estimate engine and transmission health, very costly to replace these if they fail, and very easy for a seller to fool buyers by for example pouring oil into spark plug holes to prevent engine damage from being audible.