A time gun seems like a bad weapon for the work they do, IMHO. The batons that mess with your personal time field, the reset bombs, and time loop collars I think fit what they need to do. A gun tied to time could end up missing the target and erasing/resetting someone or something they didn't intend to.
I think showing them outmatched is a good story beat. They probably have "the Bug Guns™️" held in reserve. Would not be surprised if Justice Peace unloads one later in the series.
I'm sure they'll be explaining why that is bad in the next episode since the other Loki just created a TON of branches with the scatter bombs.
I bet if you reset a timeline with a variant present, the variant just gets to watch it go back to normal. Then they could just start over again. That would explain why you have to "delete" or "remove" a variant physically from the timeline.
Hence why they can hide in big events and not be seen because they can just do whatever after each reset.