Steve Melton's 94th-minute goal deservedly gave the Linnets their first point at bogey side Banbury in three games.
The Walks outfit seemed headed for an FA Cup hangover in their first league game for more than a month, following the weekend's dream draw with Oldham.
However, they showed their team spirit by levelling three times and, to their credit, they persevered with their passing game on Tuesday.
Caretaker boss Shaun Carey's preparations were hampered by late arrivals due to traffic problems.
Carey said: "Jack Defty, who through no fault of his own, was late in traffic and I had to change a few things.
"I said afterwards how we battled back and We didn't throw in the towel. We always seem to start slowly there, and they scored early on to put us on the back foot.
"We dug in. Now we've got to back to what we're good at, clean sheets."
Lynn never seem to do well at the Spencer Stadium, and after just three minutes from the left of the box Jon Gardiner rifled into the net to put the Puritans ahead.
There were half-hearted penalty appeals after Michael Frew was challenged in the box, then on 39 minutes Grant Cooper against the post from a corner.
Adam Smith's 25 yard effort went inches wide of keeper Alan Judge's right post, and Matt Nolan spooned over at the opening half closed.
In the 54th minute Frew levelled after a corner was not cleared by the home defence.
Three minutes later the Puritans regained the lead when Mark Warren brought down Stuart Bridges and Gardiner bagged his second strike straight down the middle from the resulting penalty.
Banbury went 3-1 on 64 minutes ahead thanks to Gardiner's cross from the left which Howard Forinton expertly headed into the top corner of the net.
But on 73 minutes Jack Defty, who had just come off the bench, won a foul, and Smith repeated one of his free-kick specials from Saturday's famous FA Cup win to send the ball into the top of the net.
Frew hit the bar, Defty headed wide and Nolan missed a gilt-edged opportunity, and it seemed like another fruitless visit to Banbury would ensue.
But a last-ditch attack and goalmouth scramble saw Melton's effort ruled over the line by the assistant referee, to make it 15 games unbeaten away this season.
All this was watched by a scout from Windsor & Eton, Lynn's FA Trophy opponents. Maybe Oldham were watching too?
Banbury: Judge, Bridges, Kinch, Sullivan, Hayward, Pond, Stanbridge, Redknap, Forinton, Baird (Blossom 23), Gardiner (Szczukiewicz 90). Sub not used: Robinson. Booked: Blossom.
Lynn: Marshall, West, Smith, Warren, Cooper, Camm, Nolan, McMahon (Norris 81), Frew, Notman (J. Defty 65), Melton. Not used: C. Defty, Gaughran, Higgs. Booked: Warren, Melton.
Att: 420. Referee: P. Kirkup (Northampton).