Scott Dash made three changes from Saturday's defeat to AFC Dunstable, Dan Naylor, Harvey Laws and Carl Knox making way for Jack Quigley, Michael Hudson and Adam Drakulic who made his first start for Rovers.
Before the game it was 16th v 17th, an important match for both sides to find some form heading into the autumn.
The first half hour, both teams mustered opportunities to open the scoring, but no one from either side seemed to have their shooting boots on. That was until the 36th minute, Michael Hudson found Caleb Steele to win his first and second header before linking with Drakulic. Steele was then brought down as he was about to pull the trigger.
Hayford took charge and sent the Hertford 'keeper the wrong way to put Rovers in the ascendency.
Rovers had their tails up, and deservedly doubled their lead just before the half time whistle. Once again, Steele was involved, as his free kick found the head of Manangu who flicked on to Hayford to show great composure to double his and his team's lead.
As the U11 Girls enjoyed their match at the half time interval, Rovers seemed comfortable in the game and were expected to come out and extend their lead in the second 45. However, the visitors started better and had the chance to pull one back seven minutes into the second half. Mohamed Amghar was adjudged to have pulled back a Hertford player from a free kick, resulting in a penalty.
Serge Erkarsian stepped up and went to Hudson's left, the Rovers stopper guessed correctly and got a good contact with the ball, but it bounced into the roof of the net.
Just three minutes later, the visitors were given more confidence. Amghar caught the Hertford winger whilst on a breakaway, and the referee did not hesitate to show the Rovers centre-half a second yellow. Scott Dash immediately brought on Bennie Boateng for Christopher Manangu as the striker was sacrificed.
Hertford were expectedly having more of the ball, and Dash made his second change of the game half way through the half as Adam Drakulic made way for Reserve team regular Taylor Bakogeorge, who had previously scored in the FA Cup match v Whitton United earlier in the season.
A few minutes later Rovers regained their two goal advantage in spectacular style. Hayford once again involved, protecting the ball well on the edge of the box and setting Harry Stratton who drilled a piledriver into the far corner, sending the Rovers fans and bench into delirium!
With Hertford beginning to chase the game, gaps appeared for Hayford and Bakogeorge to exploit on the counter, and the two of them were heavily involved in Rovers' fourth on 73 minutes. Hayford threading the ball through to Bakogeorge on the near touchline, before the substitute's direct run at goal ended with a square ball for Jack Quigley to tuck into an empty net and open his Kempston account.
The midfielder was then replaced by Dan Naylor with ten minutes to go as Rovers look to keep their lead for the remainder of the game.
Hertford did eventually manage to find a breakthrough, Jack Hewitt finding space in the box with five minutes to go who headed past Hudson from 10 yards out.
Rovers were dropping, and 5 minutes into the 6 minutes added, Hertford got their third. A cross from the right was initially headed onto the crossbar, but the rebound fell kindly to Bakarr to head into an empty net.
Rovers held out for the remaining minute and moved up to 14th with the win. Next up is another home game this Saturday, as playoff placed Didcot Town travel to Kempston.