So the directors and the writers disagree on the time travel of the movie.
Joe says Cap was in an alternate timeline.
Markus and McFeely
say differently, which is what I assumed had happened all along (hence my confusion earlier when people said how did cap get back).
Markus and McFeely accept that different people will have different viewpoints on this topic. But, in their minds, Steve was Peggy's husband all along. ‘It was always our intention that he was the father of those two children. But again, there are time travel loopholes for that,’ said McFeely. Added Markus: ‘It does introduce the idea that there are two children who have somewhat super soldier DNA.’
Dueling theories (directors vs writers), both think they're correct. I'm going with the writers on this though. *shrugs*