It seems like their defense for not being able to recall details, when the victim does have a vivid memory of what happened is going to have to be, "I do not recall", which will lead to a line of questioning about "Is your memory bad because you were drunk? Like you were drunk ALL these other times by your own admission, and like you spent most of your young adult life? Could there be other women you don't recall assaulting, because it seems like you were asking people to stop their spouses from talking after some boat trips..."
Even if the public doesn't perceive him as an attempted rapist, they're gonna get the idea that he's a drunken fratboy with terrible opinions about women. And that's still not a good luck for a Supreme Court Justice, or that the GOP would back someone like that for the highest court in the land. Politically, it might be better if he withdraws.