Most West Nile virus (WNV) infections are asymptomatic, but some lead to neuroinvasive disease with symptoms ranging from disorientation to paralysis and death. Evidence from animal models suggests that neuroinvasive infections may arise as a consequence of impaired immune protection. However, other data suggest that neurologic symptoms may arise as a consequence of immune mediated damage. We demonstrate that elevated immune responses are present in neuroinvasive disease by directly characterizing WNV-specific T cells in subjects with laboratory documented infections using human histocompatibility leukocyte antigen (HLA) class II tetramers. Subjects with neuroinvasive infections had higher overall numbers of WNV-specific T cells than those with asymptomatic infections. Independent of this, we also observed age related increases in WNV-specific T cell responses. Further analysis revealed that WNV-specific T cell responses included a population of atypically polarized CXCR3+CCR4+CCR6- T cells, whose presence was highly correlated with neuroinvasive disease. Moreover, a higher proportion of WNV-specific T cells in these subjects co-produced interferon-γ and interleukin 4 than those from asymptomatic subjects. More globally, subjects with neuroinvasive infections had reduced numbers of CD4+FoxP3+ Tregs that were CTLA4 positive and exhibited a distinct upregulated transcript profile that was absent in subjects with asymptomatic infections. Thus, subjects with neuroinvasive WNV infections exhibited elevated, dysregulated, and atypically polarized responses, suggesting that immune mediated damage may indeed contribute to pathogenic outcomes.