This answer was written before the question was changed to specifically ask about landfills in developing countries.
tl;dr: No, because landfills are the best place to capture carbon escaping from degradable materials, prevent toxic chemicals from entering the groundwater, and to store non-degradable materials.
Capturing landfill gas
According to the U.S. Environmental Protection Agency, nearly 60% of the municipal solid waste (MSW) that is landfilled in the U.S. consists of degradable materials such as paper, food, wood, and yard waste:
All of these materials will eventually biodegrade into methane and carbon dioxide. In a managed, anaerobic landfill with methane capture, nearly all of the methane can be captured and used to generate electricity. If such waste were left as litter, these gases would escape directly to the atmosphere, contributing to the greenhouse effect without doing any useful work along the way.
In the U.S., methane capture is required at most landfills, and the EPA has an active program to institute methane capture at eligible landfills. Per the EIA, landfill gas (LFG) projects in the U.S. in 2020 captured a quantity of methane that, if released to the atmosphere, would represent 100.4 million metric tons of CO2 equivalent (MMTCO2e). This is slightly more than than the 99.4 MMTCO2e released from landfills without LFG capture, according the the EPA.
Treating leachate
Most LFG collection systems have the benefit of combing with a leachate collection system (image from EIA):
Leachate is the nasty mix of liquids and chemicals that ooze out of garbage. It oozes out of all garbage wherever it is, so better to collect it in a landfill where it can be treated, which is typically done in large tanks with the same kind of enzymes and neutralizing chemicals that are used to treat sewage.
The leachate itself is actually a very corrosive substance, which means that it helps break down the stuff in the landfill, increasing methane production and reducing the time it takes for things to break down.
All the other stuff
As for everything else (mostly metal, glass, and plastic) it originally came from a hole in the ground, so putting back in one seems appropriate.