There's an everlasting myth that it is possible to remain friends with an ex (for this blame a certain couple of Friends named Ross and Rachel).
In reality it's not true, at least certainly not to begin with. Delete their number, all texts, absolutely anything physical linking you to them.
However, do not completely erase the memory of them; take heed in the lesson from 'Eternal Sunshine of the Spotless Mind'.