Because, just like with a handful of shirts, with a handful of men or women, you will tend to get confused.
You can also use relationship to describe dealings between various groups, as “Members of the European Union have a relationship of economic interconnectedness,” or “A lease helps define your relationship with your landlord.” You'll also see relationship used with interactions that don't directly involve people.
In many of these cases, the word relation is better.
For example, in the sentence "We are always looking at the relationship between ticket price and sales," it would be better to use relation.
“I don’t want to have this conversation,” he said with conviction. And compromise is a thing that people in relationships do.”“I don’t even know if I want to have a relationship with you, I just want to date you some more and find out,” I shot back.
How presumptuous of him to assume I wanted a relationship with him, not to mention how confounded I was by his jump in logic.