There may be some lingering resentment.
People sometimes hang hunting or finishing taxidermy trophies of their kills/catches on their walls. The taxidermy has come to life and bit the man who was the one who killed him. It's thus understandable that the fish (if it could come alive) would resent its killer; the man nonchalantly quips about their tenuous relationship.