Because you're a stupidly endearing person, maybe?

I mean, part of it is he doesn't like being hurt and he doesn't trust you to protect yourself, but there is an undeniable sense of care in how he talks about you. Whatever reason, he is legitimately fond of you as well.

But just as me getting in a fight with him because I worried about you doesn't make it your fault, him caring about you doesn't make his actions your fault.