One of the all-time classic cup clashes saw Khalsa snatch an unlikely victory at the death after both sides relinquished two-goal leads.
Khalsa raced into a lead on nine minutes on a windy and challenging day to play football. The ball ping ponged around the area before Tes Robinson headed back across goal and Josh McKenzie nodded the ball into the net.
It looked like Khalsa were cruising but on 29 minutes Ryan Seal fed Charlie Evans who took advantage of some hesitancy to round Sam Arnold and poke the ball into the goal.
Three minutes later the lead had evaporated completely. Seal was again involved, putting in a wicked corner which was headed by Loyiso Recci, saved well by Arnold but then headed in at close range by Rico Alexander.
Things went from bad to worse on 42 minutes, Kieran Fitzgerald finishing smartly from long distance after Arnold raced out to half-heartedly clear Matt Hill’s long ball forward.
The second half saw Rugby stay on top and go two ahead on 56 minutes, Fitzgerald given far too much time and space to slam home his goal and make it 4-2.
And Williams was to prove pivotal, making it 4-3 with his first touch from a probing free kick.
Seven minutes later and Williams pulled Khalsa level, turning and shooting into the roof of the net to set up an exciting last 15 minutes.
McGrady almost put Khalsa ahead again on 77 minutes, Matt Hill saving well for Rugby low and to his right.
But in stoppage time, McKenzie smashed home after brilliant work by McGrady to complete a famous comeback and put Khalsa into the next round.
Sporting Khalsa: Sam Arnold, Carlo Franco, Lewis Worsey, Leon Miles, Tes Robinson (captain), Alex Perry (Alex Woodhouse), Kieron Whittaker, Liam Armstrong, Josh McKenzie, Dan Westwood (James McGrady), Gaz Singh (Sean Williams). Subs not used: Dave Meese, Mike Perks.