Knowing whether an object is owned and by whom is essential to avoid costly conflicts. We hypothesize that everyday interactions around objects are influenced by a minimal sense of object ownership grounded on respect of possession. In particular, we hypothesize that tracking object ownership can be influenced by any cue that predicts the establishment of individual physical control over objects. To test this hypothesis we used an indirect method to determine whether visual cues of physical control like spatial proximity to an object, temporal priority in seeing it, and touching it influence this minimal sense of object ownership. In Experiment 1 participants were shown a neutral object located on a table, in the reaching space of one of two characters. In Experiment 2 one character was the first to find the object then another character appeared and saw the object. In Experiments 3 and 4, spatial proximity, temporal priority, and touch are pitted against each other to assess their relative weight. After having seen the scenes, participants were required to judge the sensibility of sentences in which ownership of the object was ascribed to one of the two characters. Responses were faster when the objects were located in the reaching space of the character to whom ownership was ascribed in the sentence and when ownership was ascribed to the character who was the first to find the object. When contrasting the relevant cues, results indicate that touch is stronger than temporal priority in modulating the ascription of object ownership. However, all these effects were also influenced by contextual social cues like the gender of both characters and participants, the presence of a third-party observer, and the co-presence of characters. Consistently with our hypothesis, results indicate that many different cues of physical control influence the ascription of ownership in daily social contexts.