Wednesday's talks were the first high-stakes meeting between the government of President Emmanuel Macron and union bosses since nationwide protests and strikes against the legislation started in mid-January.
The talks, ahead of a new day of walkouts and marches planned for Thursday, lasted about an hour. Union leaders were united in saying the meeting had been a failure.
"We decided to put an end to this useless meeting...The prime minister has chosen to send us back to the streets," said Sophie Binet, the newly elected first woman leader of the hardline CGT union, France's second biggest.
The government says it is necessary to raise the retirement age to 64 from 62 for most workers to balance the pension budget in years to come. The unions say the money can be found elsewhere.
Borne told reporters that she was open to further talks but that she would not back down on the retirement age.
CGT activists hung a banner reading "64, it's no" from the top of the capital's Arc de Triomphe monument.
Laurent Berger, head of the more moderate CFDT trade union, the country's largest, called for workers to join en masse Thursday's strikes and street protests.
He said the vast majority of people opposed the pension system change and that public opinion was not shifting. The unions' determination to fight it was unwavering, he added.
Labor representatives complain they are not being listened to despite weeks of protest marches and unrest against the pension overhaul, Reuters reported.
Macron's decision to ram through the legislation without a final vote in parliament last month only added to the fury of unions and the wider public.
Opponents of the bill will now be waiting for the Constitutional Council to give its verdict on the bill on April 14. The council has the power to strike down the bill - or parts of it - if it considers it breaches the constitution, but it rarely rejects entire bills.
MP/PR