Craig had given Harry Hamilton his first appearance of the season in the second row, while Kenny Harper replaced Alan Laidlaw in the pack as captain Garry Cockburn again remained at hooker.
Behind the scrum, Doug Critchton took over the number nine jersey, while Euan Herkes returned in midfield and Rob McLuckie took over on the right wing.
The Reds came out of the blocks at 100-miles-an-hour, and James Ferguson produced a virtuoso opening half-hour, as he stole the show.
The full-back crossed three times inside the first half as the Reds took control of the match, and looked to put their recent troubles behind them.
Armandt Pryor impressed again in the back-row, while Kyle Adams was also looking lively out wide for Bryan Craig's side.
With less than 10 minutes to go until the break, and the Reds seemingly cruising towards their first league win of the season, the Fife outfit got themselves back into the game.
Pryor was sent to the sin-bin, and from a quick-tap penalty, Dayle Turner raced over to reduce the arrears (19-5).
The visitors struck again while the Reds' South African was still in the bin, as scrum-half Gavin McKenzie evaded a number of would-be tacklers to go over for Kirkcaldy's second try.
Early dominance from the home side, who had just fallen away towards the interval as the visitors' pack began to get in the game, but still, Bryan Craig's side turned around in front.
HALF-TIME: Haddington 19-12 Kirkcaldy
The Reds did not start the second half well, as Finlay Smith added a penalty to his earlier conversion to further reduce the gap, before that man Turner was to the fore again for the visitors.
He linked well with his colleagues before going over, and allowing Smith to kick the visitors ahead at 19-22.
The drama continued at Neilson Park as the prolific flying machine Adams struck again, his fourth in as many league games this season, after good work between Ferguson and Crichton, to secure the four-try bonus point and put the Reds back in front.
That try was then topped by one of the finest sibling combinations Neilson Park has seen for many years.
The Reds worked the ball wide, and with captain Garry Cockburn in midfield and only his brother Calum on the wide expanses, the skipper put boot to ball and landed a perfectly-weighted cross-field kick into his Calum's arms, allowing the big centre to beat three covering defenders and score the club's fifth try of the match.
But it was not to be enough, as Chris Mann's try at the death snatched victory away from the home side, who will be impressed with their performance, if not the result.