UK PM Keir Starmer intends to "push on" with the deal to hand over the Chagos Islands, Navin Ramgoolam says.