What does it mean to be agent and how do we perceive others as such? Do the same rules apply when interacting with others who are radically different from ourselves? We typically perceive the agency of others through their behavior, as they engage various aspects of their affordance field. The affordance concept refers to an organism's environmentally anchored action possibilities, but questions abound as to how more precisely to understand the relational, modal, future-directed and dynamic aspects of this notion. These complexities might be seen as intensified in social interaction where we also might perceive others' agency through reciprocal negotiations and sharing of affordances. Via Merleau-Ponty's analysis of social perception I try to bring together a re-interpretation of affordances and of perceptible agency, which might begin to give us some tools to understand interactions with agents truly other than ourselves.