How would I make a hostile entity neutral if a player is holding something in their offhand?

I am trying to make an entity stop being hostile if a player holds a certain item, but everytime I try it, they would either become fully passive, or fully hostile, I tried using conditions on the Attack Players on sight block but it doesn't work, help?