Bystander activation of memory T cells occurs in the absence of cognate antigen during infections that elicit strong systemic inflammatory responses, which subsequently affect host immune responses. Here we report that memory T cell bystander activation is not limited to induction by systemic inflammation. We initially observe potential T cell bystander activation in a cohort of human vaccine recipients. Using a mouse model system, we then find that memory CD8+ T cells are specifically recruited to sites with activated antigen-presenting cells (APCs) in a CXCR3-dependent manner. In addition, CXCR3 is also necessary for T cell clustering around APCs and T cell bystander activation, which temporospatially overlaps with the subsequent antigen-specific T cell response. Our data thus suggest that bystander activation is part of the initial localized immune response, and is mediated by a site-specific recruitment process of memory T cells.