What if they do this in the style of "Beowulf"?
Maybe just for Bilbo? That way Ian Holm would still play him, but would be younger than he actually is. Just have a Serkis or someone else do the on set work, and CGI over them with a young Ian.
What about Hugo Weaving as Elrond again. Elrond was in the Hobbit right? It has been a while since I read it.
I honestly can't imagine it would be difficult to get any of the original cast back to reprise their roles. They all seemed to have loved their LotR experience. This isn't Star Wars afterall