Jacob Lindgren gave up a solo home run in the eighth inning to allow the Nationals to tie it after Nathan Eovaldi gave the Yanks a strong start going seven innings while allowing just two runs with four strikeouts.
Gio Gonzalez kept the Yankees bats silent for six innings, the Bronx Bombers caught fire in the seventh inning.
Brendan Ryan, who was making his season debut after spending the first two months on the disabled list, put the Yankees on the board in the seventh with an RBI triple. Later in the inning Alex Rodriguez, Brett Gardner and Brian McCann each added RBI base hits to put the Yankees ahead 4-2.
After rallying to take a two run lead into the eighth, Jacob Lindgren relieved Eovaldi and surrendered a two run home run to Michael Taylor that tied the score at 4-4.
The Yankees have an off day tomorrow before beginning a weekend series against the Orioles in Baltimore on Friday night. Michael Pineda (7-2, 3.33 ERA) will be on the mound for the Yankees while the Orioles have not yet announced a starter.