My custom plant only drops its loot when the block under it is broken?

I've made a custom plant as well as a custom drop, the loot table has it drop itself as well as its fruit, but when you punch the plant it drops nothing. It only drops loot when you break the dirt block under it, and I can't find the setting that would change that. Anybody know what I'm talking about or experience something similar? It's my first time making mods or coding of any kind, so sorry if it has an obvious answer that flew over my head >_<