It really comes down to what a fusion is supposed to be - it is supposed to be much stronger than it's component members together.
Goku at MUI casually toys with Jiren, so he's not unlimited. Vegeta at blue-er-er is beyond GoD level. Goku and Vegeta should both be at that level. If they fuse, it should be some level of exponential growth from there.
I don't think that Jiren is exponentially stronger than an un-exhausted Goku or Vegeta. Stronger definitely. But not "limitlessly" stronger.
As for judging what Gogeta does to Broly, well, you also have to remember that Gogeta doesn't want to kill Broly, only beat him.
All of that said, this isn't an argument I'd bet the rent on or anything.
Edit: this was a reply to Arby's question on the first page, btw.