In addition to entering a Leaving Date, you will need to off-roll the student.
Pop to the extended tab of their profile and scroll to the very bottom. You will see a little blue button labeled Off Roll - click this.
You will then be asked to enter a Leaving Reason, so choose the relevant one for the student and click Complete Off Roll.
This student is now off-roll and will not show in registers. To find former pupils go to Admin > Students > Search not On Roll.