The perfect time for a recast was First Class. Had Jackman not been in that film, they could have started a new story that didn't connect with the Singer films, but they had to put that funny, but pointless Wolverine cameo in the film, and as a result First Class became a prequel, rather than a reboot. It's fine, it gave us DOFP, but now we have an almost 50 year old Jackman/Logan and a bunch of young actors wearing makeup to look older in Apocalypse, rather than doing several Xmen films and stories in the 1960's and 70's with Magneto and the X men all in their prime. They are wasting the young cast by making one film/story per decade and aging these characters/young actors.
Some people say that Jackman IS Wolverine and they call him the definitive version, but I can't say that because unlike Superman or Batman, no one else has played Wolverine. With Batman and Superman you pick and choose and debate which actor is better or who is the definitive version, but not with Wolverine, which is why I'm looking forward to seeing another actor play that role. I like Jackman, but then again, he's all I know. Now, is not the time to recast, because of Apocalypse, so we'll see where they go after the next film, the 1990's?