Inclusion Criteria:

   - INCLUSION CRITERIA FOR STRATA A, B, D AND E

   - Tumor: patient must have one of the following diagnoses to be eligible:

   - Stratum A, currently closed to enrollment: Patients must have a recurrent, progressive
   or refractory DIPG following radiation therapy with or without chemotherapy

      - Histologic diagnosis is not required for patients with typical imaging findings
      of DIPG (defined as patients with a diffuse expansile mass centered in and
      involving at least 2/3 of the pons); patients with brainstem tumors who have
      undergone biopsy with a diagnosis of high-grade glioma or diffuse infiltrating
      glioma are also eligible

   - Stratum B: Patients must have a histologically confirmed diagnosis of a non-brainstem
   high-grade glioma (NB-HGG) that is recurrent, progressive or refractory following
   therapy which included radiotherapy; spinal primary disease is eligible

   - Stratum D: Patients must have a histologically confirmed diagnosis of ependymoma that
   is recurrent, progressive or refractory following therapy which included radiotherapy

   - Stratum E: Patients must have a histologically confirmed diagnosis of medulloblastoma
   that is recurrent, progressive or refractory following therapy which included
   radiotherapy

   - Patients must have adequate pre-trial formalin-fixed paraffin-embedded (FFPE) tumor
   material available for use in the biology studies mutational analysis and genome wide
   sequencing for each stratum

      - Patients with DIPG who have tissue available are requested to submit similar
      tissue as patients in other strata; however, this is not required for eligibility

   - All subjects must have measurable disease in 2-dimensions on MRI scan of the brain;
   disease should be consistently measured with the two largest perpendicular dimensions

   - Patients must have received prior radiation therapy and/or chemotherapy and recovered
   from the acute treatment related toxicities (defined as =< grade 1 if not defined in
   eligibility criteria) of all prior chemotherapy, immunotherapy or radiotherapy prior
   to entering this study; there is no upper limit to the number of prior therapies that
   is allowed

   - Patients must have received their last dose of known myelosuppressive anticancer
   therapy at least three (3) weeks prior to study enrollment or at least six (6) weeks
   if prior nitrosourea

   - Biologic or investigational agent (anti-neoplastic): Patient must have received their
   last dose of the investigational or biologic agent >= 7 days prior to study enrollment

      - For agents that have known adverse events occurring beyond 7 days after
      administration, this period must be extended beyond the time during which adverse
      events are known to occur; the duration must be discussed with and approved by
      the study chair

      - Monoclonal antibody treatment and/or agents with prolonged half-lives: at least
      three half-lives must have elapsed prior to enrollment

   - Patient must have completed immunotherapy (e.g. tumor vaccines, oncolytic viruses,
   etc.) at least 42 days prior to enrollment

   - Patients must have had their last fraction of:

      - Craniospinal irradiation >= 3 months prior to enrollment

      - Other substantial bone marrow irradiation >= 6 weeks prior to enrollment

      - Local palliative radiation therapy (XRT) (small port) >= 2 weeks

   - Patient must be >= 12 weeks since autologous bone marrow/stem cell transplant prior to
   enrollment

   - Patients must be fully recovered from all acute effects of prior surgical intervention

   - All races and ethnic groups are eligible for this study

   - Patients with neurological deficits should have deficits that are completely stable
   for a minimum of 1 week (7 days) prior to enrollment

   - Karnofsky performance scale (KPS for > 16 years of age) or Lansky performance score
   (LPS for =< 16 years of age) assessed within two weeks of enrollment must be >= 70;
   patients who are unable to walk because of neurologic deficits, but who are up in a
   wheelchair, will be considered ambulatory for the purpose of assessing the performance
   score

   - Absolute neutrophil count >= 1000 cells/uL

   - Platelets >= 75,000 cells /uL (unsupported, defined as no platelet transfusion within
   7 days)

   - Hemoglobin >= 8 g/dl (may receive transfusions)

   - Total bilirubin =< 1.5 times institutional upper limit of normal (ULN)

   - Alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase [SGPT]) =< 3 x
   institutional upper limit of normal

   - Albumin >= 2 g/dl

   - Serum creatinine based on age/gender as noted below; patients that do not meet the
   criteria below but have a 24 hour creatinine clearance or glomerular filtration rate
   (GFR) (radioisotope or iothalamate) >= 70 ml/min/1.73 m^2 are eligible

      - 1 to < 2 years: 0.6 mg/dl

      - 2 to < 6 years: 0.8 mg/dl

      - 6 to < 10 years: 1 mg/dl

      - 10 to < 13 years: 1.2 mg/dl

      - 13 to < 16 years: 1.5 mg/dl (male), 1.4 mg/dl (female)

      - >= 16 years: 1.7 mg/dl (male), 1.4 mg/dl (female)

   - Pulse oximetry > 93% on room air and no evidence of dyspnea at rest

   - Patients must be off all colony-forming growth factor(s) for at least 1 week prior to
   registration (e.g. filgrastim, sargramostim, erythropoietin); 2 weeks must have
   elapsed for long-acting formulations

   - Patients must be willing to use brief courses (at least 72 hours) of steroids as
   directed for potential inflammatory side effects of the therapy if recommended by
   their treating physician

   - Female subjects of childbearing potential must not be pregnant or breast-feeding;
   female patients of childbearing potential must have a negative serum or urine
   pregnancy test within 72 hours prior to receiving the first dose of study medication;
   if the urine test is positive or cannot be confirmed as negative, a serum pregnancy
   test will be required; pregnant women are excluded from this study; breastfeeding
   should be discontinued if the mother is to be treated with MK-3475 (pembrolizumab)

   - Patients of childbearing or child fathering potential must be willing to use 2 methods
   of birth control or be surgically sterile or abstain from heterosexual activity while
   being treated on this study and for 6 months after the last dose of study medication

   - The patient or parent/guardian is able to understand the consent and is willing to
   sign a written informed consent document, inclusive of assent where appropriate,
   according to institutional guidelines

   - INCLUSION CRITERIA FOR STRATUM C: Diagnosis of hypermutated brain tumors Patients with
   brain tumors and increased tumor mutation burden as determined by

      - Confirmed diagnosis of CMMRD syndrome by Clinical Laboratory Improvement Act
      (CLIA)-certified germline gene sequencing OR

      - Confirmation of high mutation burden by whole genome/exome sequencing performed
      in a CLIA-certified laboratory and/or the use of Foundation One next generation
      sequence panel or another CLIA approved targeted sequencing lab with publicly
      available correlations between number of mutations found in the panel and
      mutations per megabase and/or genome; for protocol purposes a high mutation
      burden will be defined as at least 100 non-synonymous coding-region mutations by
      whole exome/genome sequencing (well above two standard deviations of the number
      of median similar mutations described in pediatric CNS cancers) AND/OR a high
      tumor mutation burden (TMB) or intermediate TMB based on the reporting parameters
      of the panel; TMB parameters provided for the Foundation One reports are high
      tumor mutation burden is >= 20 mutations per megabase or intermediate TMB is
      between 6 to 19 mutations per megabase

      - Confirmed diagnosis of Lynch syndrome by CLIA-certified germline gene sequencing;
      patients with Lynch syndrome will not be accounted for in primary objective
      unless their tumors are determined to have the minimum number of mutations
      described above but they will still be eligible for this study

      - Low-grade tumors in patients with CMMRD or Lynch syndrome do not have to reach
      the threshold of 100 mutations for study inclusion

   - INCLUSION CRITERIA FOR STRATUM C: Patients must have a histologically confirmed
   primary brain tumor that is recurrent, progressive or refractory; inclusion criteria
   encompasses all types of brain tumors (e.g. gliomas, embryonal tumors or any other
   type of brain tumor as long as other eligibility criteria are met;

      - Patients with high-grade gliomas are eligible for this clinical trial at least 2
      weeks after completion of radiotherapy independent of tumor
      progression/recurrence as long as they are not enrolled on any other therapeutic
      clinical trial and there is macroscopic residual disease

      - Patients with other concomitant tumors associated with CMMRD syndrome including
      gastrointestinal polyps/adenomas and carcinomas, lymphomas and leukemias will be
      eligible as long as they are not requiring anticancer therapy directed against
      these other cancers and meet all other eligibility criteria

   - INCLUSION CRITERIA FOR STRATUM C: Patients must have adequate pre-trial FFPE tumor
   material available and be willing to provide a blood sample for use in the genome wide
   sequencing studies; while tissue is required for genome-wide sequencing of tumor and
   germline samples, patients will be deemed eligible for the study with a minimum of
   approximately 10 unstained slides for the planned analysis

   - INCLUSION CRITERIA FOR STRATUM C: All subjects must have measurable disease in
   2-dimensions on MRI scan of the brain and/or spine; disease should be consistently
   measured with the two largest perpendicular dimensions

   - INCLUSION CRITERIA FOR STRATUM C: Patients must have received prior radiotherapy
   and/or chemotherapy with the following exceptions:

      - Patients with secondary CNS cancers after a previous medical problem/malignancy
      who cannot receive full dose of radiotherapy (> 50 Gy) as long as they meet all
      other eligibility criteria

      - Patients with progressive low-grade gliomas and CMMRD or Lynch syndrome Patients
      must have recovered from the acute treatment related toxicities (defined as =<
      grade 1 if not defined in eligibility criteria) of all prior chemotherapy,
      immunotherapy or radiotherapy prior to entering this study; there is no upper
      limit to the number of prior therapies that is allowed

   - INCLUSION CRITERIA FOR STRATUM C: Patients must have received their last dose of known
   myelosuppressive anticancer therapy at least three (3) weeks prior to study enrollment
   or at least six (6) weeks if prior nitrosourea

   - INCLUSION CRITERIA FOR STRATUM C: Patient must have received their last dose of the
   investigational or biologic agent >= 7 days prior to study enrollment

      - For agents that have known adverse events occurring beyond 7 days after
      administration, this period must be extended beyond the time during which adverse
      events are known to occur; the duration must be discussed with and approved by
      the study chair

      - Monoclonal antibody treatment and/or agents with prolonged half-lives: at least
      three half-lives must have elapsed prior to enrollment

   - INCLUSION CRITERIA FOR STRATUM C: Patient must have completed immunotherapy (e.g.
   tumor vaccines, oncolytic viruses, etc.) at least 42 days prior to enrollment

   - INCLUSION CRITERIA FOR STRATUM C: Patients must have had their last fraction of:

      - Craniospinal irradiation >= 3 months prior to enrollment

      - Other substantial bone marrow irradiation >= 6 weeks prior to enrollment

      - Local palliative radiation therapy (XRT) (small port) >= 2 weeks

   - INCLUSION CRITERIA FOR STRATUM C: Patient must be:

      - >= 12 weeks since autologous bone marrow/stem cell transplant prior to enrollment

      - >= 5 years since allogeneic bone marrow transplant prior to enrollment with no
      evidence of active graft versus (vs.) host disease

   - INCLUSION CRITERIA FOR STRATUM C: Patients must be fully recovered from all acute
   effects of prior surgical intervention

   - INCLUSION CRITERIA FOR STRATUM C: All races and ethnic groups are eligible for this
   study

   - INCLUSION CRITERIA FOR STRATUM C: Patients with neurological deficits should have
   deficits that are completely stable for a minimum of 1 week (7 days) prior to
   enrollment

   - INCLUSION CRITERIA FOR STRATUM C: Karnofsky performance scale (KPS for > 16 years of
   age) or Lansky performance score (LPS for =< 16 years of age) assessed within two
   weeks of enrollment must be >= 60; patients who are unable to walk because of
   neurologic deficits, but who are up in a wheelchair, will be considered ambulatory for
   the purpose of assessing the performance score

   - INCLUSION CRITERIA FOR STRATUM C: Absolute neutrophil count >= 1000 cells/uL

   - INCLUSION CRITERIA FOR STRATUM C: Platelets >= 75,000 cells/uL (unsupported, defined
   as no platelet transfusion within 7 days)

   - INCLUSION CRITERIA FOR STRATUM C: Hemoglobin >= 8 g/dl (may receive transfusions)

   - INCLUSION CRITERIA FOR STRATUM C: Total bilirubin =< 1.5 times institutional upper
   limit of normal (ULN)

   - INCLUSION CRITERIA FOR STRATUM C: ALT (SGPT) =< 3 x institutional upper limit of
   normal

   - INCLUSION CRITERIA FOR STRATUM C: Albumin >= 2 g/dl

   - INCLUSION CRITERIA FOR STRATUM C: Serum creatinine based on age/gender as noted below;
   patients that do not meet the criteria below but have a 24 hour creatinine clearance
   or GFR (radioisotope or iothalamate) >= 70 ml/min/1.73 m^2 are eligible

      - 1 to < 2 years: 0.6 mg/dl

      - 2 to < 6 years: 0.8 mg/dl

      - 6 to < 10 years: 1 mg/dl

      - 10 to < 13 years: 1.2 mg/dl

      - 13 to < 16 years: 1.5 mg/dl (male), 1.4 mg/dl (female)

      - >= 16 years: 1.7 mg/dl (male), 1.4 mg/dl (female)

   - INCLUSION CRITERIA FOR STRATUM C: Pulse oximetry > 93% on room air and no evidence of
   dyspnea at rest

   - INCLUSION CRITERIA FOR STRATUM C: Patients must be off all colony-forming growth
   factor(s) for at least 1 week prior to registration (i.e. filgrastim; sargramostim;
   erythropoietin); 2 weeks must have elapsed for long-acting formulations

   - INCLUSION CRITERIA FOR STRATUM C: Patients must be willing to use brief courses (at
   least 72 hours) of steroids as directed for potential inflammatory side effects of the
   therapy if recommended by their treating physician

   - INCLUSION CRITERIA FOR STRATUM C: Female subjects of childbearing potential must not
   be pregnant or breast-feeding; female patients of childbearing potential must have a
   negative serum or urine pregnancy test within 72 hours prior to receiving the first
   dose of study medication; if the urine test is positive or cannot be confirmed as
   negative, a serum pregnancy test will be required; pregnant women are excluded from
   this study; breastfeeding should be discontinued if the mother is to be treated with
   MK-3475 (pembrolizumab)

   - INCLUSION CRITERIA FOR STRATUM C: Patients of childbearing or child fathering
   potential must be willing to use 2 methods of birth control or be surgically sterile
   or abstain from heterosexual activity while being treated on this study and for 6
   months after the last dose of study medication

   - INCLUSION C