The 21-year-old Holliday ripped a frozen rope to the right field corner, scoring the speedy Dylan Carlson all the way home from first base.
Baltimore put an end to the Seattle Mariners' seven-game win streak. The two teams will play the final game of the series on Thursday afternoon.
Despite the Orioles' disappointing season, Holliday himself has been having a much better year compared to his rookie campaign, slashing .250/.301/.390 with 14 home runs and 45 RBI. Now, he can also add his first career walk-off hit.
Baltimore held a 3-1 lead heading into the top of the ninth inning, but after Josh Naylor scored Julio Rodriguez on a ground out, Dominic Canzone's sac-fly tied it all up heading into the bottom of the ninth inning, where the Orioles magic finally pulled off a trick.
Seattle falls to 67-54 on the season as they are now, once again, a game back of the Houston Astros for first place in the American League West division. Meanwhile, the 54-66 Orioles continue to be near the bottom of the league, but a walk-off win always feels good.