Damage A Specific Item Among Item Stack

Hello, I'm currently having trouble damaging an item in an item stack.

Basically, I've made an item that has a maximum stack of 1 and when holding it in the off-hand, it will damage it. It works, but it also deals damage to the other same items in the player's inventory. How can I fix that ? 

Thanks in advance.