While Natsume's background wasn't always poor—well, I say "always" because he has spent most of his life passed around from household to household, heaped upon and then sent away by distant relatives. There were so many homes that they lent many different experiences, but it's made clear that the majority of these families weren't emotionally kind... and several of them weren't physically kind, either. This is some stuff he's never addressed with anyone else or even himself, but I think a big part of that emotional denial is he hasn't spoken with anyone else from an abusive background. I can see some potential for a slowburn friendship here.