Mahathir, 97, who served as Malaysia's prime minister for more than two decades in two stints, came in fourth in a five-way fight in Langkawi, a resort island in Malaysia’s northwest, which he had won with a large majority in the previous poll in 2018.
The seat was won by Mohd Suhaimi Abdullah, a candidate from the Perikatan alliance, which is led by another former prime minister, Muhyiddin Yassin.
It was Mahathir's first election loss since 1969.
Mahathir leads a coalition that has vowed to take down the incumbent Barisan Nasional government over corruption allegations.
But his alliance is not considered a key contender, with Barisan facing two other major coalitions - Muhyiddin's bloc and another led by Mahathir's long-time rival Anwar Ibrahim.
Mahathir told Reuters in an interview this month he would retire from politics if he lost.
"I don't see myself being active in politics until I'm 100-years-old," he said. "The most important thing is to transfer my experience to the younger leaders of the party."
ZZ/PR