Pathogen-specific CD8+ T cell responses restricted by the nonpolymorphic nonclassical class Ib molecule human leukocyte antigen E (HLA-E) are rarely reported in viral infections. The natural HLA-E ligand is a signal peptide derived from classical class Ia HLA molecules that interact with the NKG2/CD94 receptors to regulate natural killer cell functions, but pathogen-derived peptides can also be presented by HLA-E. Here, we describe five peptides from severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) that elicited HLA-E-restricted CD8+ T cell responses in convalescent patients with coronavirus disease 2019. These T cell responses were identified in the blood at frequencies similar to those reported for classical HLA-Ia-restricted anti-SARS-CoV-2 CD8+ T cells. HLA-E peptide-specific CD8+ T cell clones, which expressed diverse T cell receptors, suppressed SARS-CoV-2 replication in Calu-3 human lung epithelial cells. SARS-CoV-2 infection markedly down-regulated classical HLA class I expression in Calu-3 cells and primary reconstituted human airway epithelial cells, whereas HLA-E expression was not affected, enabling T cell recognition. Thus, HLA-E-restricted T cells could contribute to the control of SARS-CoV-2 infection alongside classical T cells.