Vegans argue that most of the grain grown worldwide is used for livestock fattening. How many people could be fed using those fields if they were no longer used for growing livestock feed?
Here are some references with opposing views:
http://www.lierrekeith.com/book-ex_the-vegetarian-myth.php
http://adaptt.org/veganism.html# in section Animal Ag and Environmental Destruction
Your question seems to be specifically limited to grain fed to livestock, even though a great deal of the feed grown for livestock is not grain, and most livestock are not fed, let alone fed grain. It's likely based on stories like this one saying "U.S. could feed 800 million people with grain that livestock eat, Cornell ecologist advises animal scientists". I wonder whether in fact you are asking "is that Cornell story true"?
I suspect it is, but you have to read past the headline to get any value from the story. That link even includes useful quotes like this:
About 26 million tons of the livestock feed comes from grains and 15 million tons from forage crops.
So the question is not "if we fed people all that animal food", but "if we took the grain and fed that to people". Given that and the other reasonable points made, I am inclined to take the research at face value. The news media nonsense that flows from it, much less so. For example, anyone suggesting that the researcher assumes a vegetarian (or vegan) lifestyle would result has neither read the article nor research paper, or is lying. Or both. The vegetarian research has been done, but this is not it.
There are multiple references to US livestock consuming most of US grain production so I think that isn't controversial. Also without question, the grain could be eaten by people. I strongly suspect that the US is, however, an outlier on that aspect - worldwide, most grain will be fed to humans rather than other animals - GlobalIssues and others suggest about 1/3rd but sadly don't give references.
Globally, about 2.5 billion tons of grain are produced each year and using that 1/3rd number, approximately 800 million tons is wasted on livestock. Again, roughly speaking (using freerice.com's estimate of 0.5kg/person/day) that could support about 4 billion people, assuming they could also get enough other food to provide essential nutrients (and water etc etc).
If instead we focus on land used to grow all fodder crops the question becomes more complex. For example, fodder crops are grown on land that can't usefully be used to produce food for humans, or fodder is a secondary product of a food crop (palm oil kernals, for example). But if the secondary product wasn't sold, it's likely that less of the primary crop would be produced (at least on a global, roughly Pareto-optimal first approximation).
The second problem is that most fodder crops that can be eaten by humans are poor sources of protein, and in many ways it's protein that matters - that is the dietary factor that limits human population more than any other (the political factors that lead to calory deficiency are not really part of the question as I understand it - we currently produce enough food calroies to feed roughly twice our population). While it's possible to grow protein crops on much fodder crop farmland, the yield will be reduced.
In the limiting case, you might look at the cutting of remote Australian grassland for hay in flood years, where that land is generally not usable at all for human feed crops. If hay is produced one year in five, perhaps a small wheat crop could be produced instead. But wheat is harder to grow and more likely to fail to produce a usable harvest. So maybe one year in 20 you'd need wheat harvesting equipment, a very long way from the next nearest source of wheat. It's never going to be economic. That's based on my observation of Australian cattle farming in northern Australia, so it's anecdotal rather than factual.
You can't just say "X kilojoules of animal fodder therefore X kilojoules of human food", let alone "therefore X kilojoules of human-accessible protein". The reason poor asians feed rice to chickens is not because they particularly value chickens as pets, it's because chickens turn otherwise hard to use rice into protein and they really need the protein. Food scraps, crop residues, anything the chicken (or pig) can eat that people can't, it all becomes usable protein once the animal eats it.
Note that this is much less than the 1/3 of ALL human food that is wasted every year, and quietly skips around the other factors that mean millions of people starve despite a global surplus of food.
I have had that discussion with Vegan friends and spent some time thinking about the flow of energy.
It starts with the sun. Photosynthesis converts the radiant energy into chemical potential energy as ATP and NADPH. Other plant processes produce sugars, oils, structural carbohydrates, and other molecules for longer term storage.
Humans and animals eat the plants and produce mostly glucose internally, which also becomes ATP and NADPH for use by the cells eventually.
If we eat the animal instead of the plant, whatever energy losses that the animal had becomes an inefficiency in the chain of energy originating from the sun. It is wasted energy. When we discard most of the animal, more energy waste occurs.
One can argue that we have to eat large amounts of plants to get the various protein amino acids we can more easily get from animal meat, but vegans know what grains, beans, and other vegetables to eat so that such arguments loose ground.
I would not venture a guess exactly how many times as much grazing land is necessary to sustain a human population of 1,000 people than land used for harvest. It would need quite a bit of study and peer review to come up with a figure that would mean anything.
I'd guess more than double without flinching. Animals waste plenty of energy digesting, walking, breathing, and regulating body temperature.
Minerals have a similar story to that of energy compounds.
I suggest fewer on the grounds that livestock consumption of feed is part of a cycle. That is, the livestock consume the feed and excrete part of that, which in turn aids the production of more feed. Without the excretion being fed back as manure in one form or another there would not be as much plant growth.
Hence, the consumption of livestock is really just a partial subtraction of the total energy of the plant growth which is continuously cycling (with the addition of sunlight and rain in each cycle of course). Take livestock out of the cycle and the growth will be retarded.
Different forms of livestock feeding practices also affect this. Where I live in NZ most livestock graze on grass year-round with the addition in Winter of mostly hay or silage (often grown on the same farm) and occasionally root crops. This could not be consumed directly by humans and so would require changes in crops from forage grasses to something else. I cannot think what this would be on a hill country farm that is not amenable to mechanised cropping but is suited to sheep grazing.
