I agree, this detour was completely pointless and took away important screen time to develop Apocalypse's agenda and his horsemen. There lies the biggest mistake in this movie for me. They should have pointed out clearly how Angel, Storm, Psylocke, and Magneto represent the victims of human oppression rising up with Apocalypse's help. Mystique and Xavier's dialogue at the mansion is hinting at that (spoiled, privilege mutants do not see the real violence still going on in the world) but it is never clearly shown. After watching it again I realized how Apocalypse is luring his horsemen in with false promises and lies ("saving" vs. "cleaning") and his first action destroying all the weapons in the world could be seen as an action of peace. But it is never really clear...
If they wanted to have a Wolverine cameo so badly, they could have introduced him as a silent henchman of apocalypse in the third act...