The clock had ticked past 90 minutes, and the five-times champions looked to be edging to a second successive disappointing draw when Marcelo floated the ball in from the left and substitute Roberto Firmino headed across to Gabriel Jesus.
Jesus's first touch took the ball away from the defender and into the path of Coutinho who raced in to poke past goal-keeper Keylor Navas from six metres out for his second goal of the tournament.
The result leaves Brazil with four points from two games after they drew 1-1 with Switzerland in their opening match, and they provisionally rise above Serbia to top Group ‘E’.
Neymar added the second with practically the last kick of the match, volleying home Jesus's square ball following a quick counter-attack as Costa Rica pushed for an equaliser.
The forward, who thought he had won a penalty in the 77th minute only for the Referee to reverse the decision after consultation with the Video Assistant Referee (VAR), was overcome by emotion on the whistle, covering his face with his hands as tears streamed down his cheeks.
That was an apt reflection of how tense the match had been for almost it's entire duration, with Costa Rica's flat-back five keeping Neymar on the periphery, as Johan Venegas and Cristian Gamboa both man-marked the forward out of the game.
Marcelo's tame shot straight at Navas in the 41st minute was the first effort on target from either side before the break.
Celso Borges spurned the best chance of the first period for Costa Rica when he ran on to Gamboa's cut back, but shot wide from eight metres out.
Brazil went into half time with the boos and whistles of their own fans ringing in their ears but emerged with a renewed sense of purpose, with Jesus heading against the bar four minutes after the restart and Coutinho sending the follow-up effort narrowly wide.
The finish was frantic as Brazil swept forward in waves, only to be repelled time and again by Costa Rica's stubborn defence, until Coutinho came to the rescue.