Joss at this point has 20 year history of doing similar things with Ray Fisher. It's well established that he was using his position to sleep with actresses working under him while on Buffy (something his wife basically said and got dismissed as a jilted ex years ago), the Charisma Carptentar story has been around for era, James Marsters has his story about Joss getting pissed that Spike was becoming a fan favorite and bullying him and pushing him against a wall, Michelle Trachenberg said after a confrontation with Whedon there was a rule that he wasn't allowed in the same room as her by themselves, Gadot talks about how bad Whedon was and how he threatened her career (former co workers also said it's a common tactic he had), Affleck outright was ready to stage a walkout.
Whedon has a history of being unprofessional and toxic and at this point on the film Ray Fisher was on, multiple big name stars have pretty much had similar issues with Whedon that at this point there's really no real reason to act like Fisher is the dude making **** up or being unreasonable.
At this point Fisher's story is either fitting a well established pattern.... or everyone over 20 years is lying about Whedon and it's some big conspiracy to take down a guy many of these people for professional reasons owe good portions of their career.