I actually wonder if Barry ever told Wally about Jay appearing during the button crossover. If he did, then it gives Wally more reason to believe Hunter is correct about the other speedsters being trapped like he was. I know Barry didn't remember Jay's name, but he did remember a strange man said he knew him just like Wally did.
And while I know it certainly isn't Barry's fault for Jay's potential return being delayed, him being all "I guess he needed a different lightning rod", and just leaving it at that is hilariously sad in hindsight. I mean, was Barry even shown to make some sort of effort to piece it together afterward?