I'm not sure why they would have played a shirts Vs. skins basketball game. As far as we're aware, they've only ever played one-on-one. Elliot, did you suggest it, by chance?
Things are confusing for Elliot, and the origins of his feelings are unclear, which is 100% intentional on my part. Maybe Elliot was born this way and has only come to realize it due to magic, maybe Elliot is only feeling certain things because magic's been an influence, or maybe it's a combination of the two.
The best one can do about when it comes to figuring out preciesly why they feel the way they do about various things is speculate. Heck, I couldn't tell you why I like half the things I do, and I don't even have the influence of magic to worry about.
At least, I assume I don't. Should probably check that.
Granted, an exception for Elliot is the mind-bending secret identity forms. That's way more clear cut. You can literally read what impact those have on various aspects of Elliot's mind (assuming you can decipher his spell book, which is so terribly written I can't even show it because nothing I could write could possibly be as incomprehensible as it's meant to be).