The temporary storage of serial order information in working memory (WM) has been demonstrated to be crucial to higher order cognition. The previous studies have shown that the maintenance of serial order can be a consequence of the construction of position markers to which to-be-remembered information will be bound. However, the nature of these position markers remains unclear. In this study, we demonstrate the crucial involvement of time in the construction of these markers by establishing a bidirectional relationship. First, results of the first experiment show that the initial items in WM result in faster responding after shorter time presentations, while we observe the opposite for items stored further in WM. Second, in the next experiment, we observe an effect of temporal cueing on WM retrieval; longer time cues facilitate responding to later WM items compared with items stored at the beginning of WM. These findings are discussed in the context of position marker theories, reviewing the functional involvement of time in the construction of these markers and its association with space.