The 35-year-old Perez had opened the scoring in the 11th minute before Bolivia fought back after the break with Juan Arce bringing them level in the 54th minute — only for Perez to strike again three minutes from the end.
Panama, who have not before featured in a Copa America and have yet to qualify for a World Cup finals, were playing in their first competitive game against a South American opponent.
“I am delighted with this win which is historic given it is our first victory in a Copa America game,” said Panama’s Colombian coach Hernan Dario Gomez.
Clearly fired up by the occasion, Panama started with real urgency. Gabriel Torres went close with a fiercely hit shot in the third minute and two minutes later Torres unleashed a dipping free-kick from 30 yards out which Bolivia keeper Carlos Lampe did well to tip over.
Perez then swung a beautiful cross-field pass into the path of Armando Cooper but the winger’s volley was high and wide.
“Los Canaleros” were clearly fired up and intent on taking the game to a nervy looking Bolivian defence and it wasn’t long before they got their reward — and in fine style.
Alberto Quintero found space on the left and whipped in a low cross which Perez met with a perfectly-timed sliding half-volley that flashed past the helpless Lampe to give Panama a fully deserved lead.
– Baldivieso under pressure –
Bolivia, whose coach Julio Baldivieso is under pressure after his team lost five of their six South American World Cup qualifiers and then crashed to a 4-0 friendly defeat to the United States last week, responded positively however.
Arce forced Panama keeper Jaime Penedo into a diving save with a nicely angled header and gradually Bolivia began to find some possession.
Perez, though, remained dangerous. The experienced target man, who plays in Major League Soccer with Vancouver Whitecaps, tested Lampe again with a snap shot but then selfishly chose to shoot from a tight position when Torres was unmarked.
Bolivia made a double substitution at the break with Fernando Saucedo and Ronaldo Eguino being replaced by Jhasmani Campos and Alejandro Melean and that seemed to give them an extra edge in midfield.
It wasn’t entirely against the run of play then in the 54th minute when Bolivia drew level — Martin Smedberg launched a free-kick from deep, into the Panama box and Harold Cummings’ attempted clearance fell to Arce who confidently fired home.
Panama thought they had restored their lead when Luis Tejada slotted home but he was narrowly ruled offside.
The pressure from Panama was mounting, however, and Quintero forced Lampe into another save and Tejada fired over the bar.
But just as it seemed the sizeable Panamanian contingent in a 13,466 crowd were going to leave frustrated, Abdiel Arroyo burst down the right flank and slipped the ball to the well-placed Perez to slot home.
Delight for Panama but for Bolivia coach Baldivieso, the result will do little to help him hang on to his job, a fact he seemed to acknowledge after the game.
“Whoever the next coach will be, if it is not me, hopefully they have the same group of talented and hard working players that I have,” he said.
