In addition to entering a Leaving Date, you will need to off-roll the student.
Pop to the extended tab of their profile and scroll to the very bottom. You will see a little blue button labeled "Off Roll" - click this.
You will then be asked to enter a Leaving Reason, so choose the relevant one for the student and click "Complete Off Roll".
This student is now off-roll and will not show in registers. To find former pupils go to Admin > Students > Search not On Roll.