Sadio Mane had fired the Reds into the lead before James Maddison levelled in front of the Kop.
Deep into time added on Liverpool were awarded a spot-kick and Milner stepped up to earn a dramatic vitory.
Here is what the media made of the victory...
Ian Doyle, Liverpool Echo
Liverpool's players danced on the field, Jürgen Klopp and his coaching staff bounced around their technical area and spilled on to the pitch, and Anfield let out a huge roar laced with ecstasy and relief. Brendan Rodgers, meanwhile, looked to the skies, a rueful look of resignation on his face. He'd seen this all before, of course. Not, though, while sat in the opposition dugout, his Leicester City beaten on his first return here in the most dramatic manner possible. And the identity of the game-changer will have hurt. Milner was one of Rodgers' parting gifts, having persuaded the midfielder to switch from Manchester City to Liverpool on the promise of featuring in his preferred central role in the summer of 2015. Within months, Rodgers was gone, Milner instead becoming a key lieutenant under Klopp, the player capable of plugging almost every hole, not least when spending a season at left-back. With Jordan Henderson benched, Milner was made skipper for the reunion with his former boss. He responded with a performance that underlined why Klopp insists there will always be a place for midfielder as long as he remains in charge. Witness the vision and execution to curl in a 50-yard pass with his unfavoured left foot to cut out Leicester centre-back Jonny Evans and send Sadio Mane sprinting through for the opener five minutes before half-time. Then there was the character to hold his nerve when presented the chance to net the winner deep into injury time by Marc Albrighton's silly foul on Mane, Liverpool having been pegged back by a Maddison strike. If Klopp has any message for Rodgers after the game, it should be thank you. For Milner, the evergreen professional who shows no signs of stopping.
Neil Jones, Goal.com
There are moments which can define a season, but you don’t expect them to arrive in October. As James Milner stood with his hands on his hips, Anfield held its collective breath. The ball was on the penalty spot, Kasper Schmeichel was stood tall, Liverpool hearts were in mouths. Cue bedlam. Their winning run continues. Leicester came to Anfield with form and confidence, Rodgers was greeted warmly. His team gave the league leaders an almighty scare, but eventually fell. Mane had some afternoon. He’d given Liverpool the lead five minutes before the break, latching onto a sublime Milner pass to slot home his 50th Premier League goal for the Reds. They have arrived in just 100 appearances. He is playing as well now as he ever has. Moved to the right wing for this game, perhaps with an eye on Leicester’s marauding left-back Ben Chilwell, Mane was in the thick of the action throughout. He made more tackles than any Liverpool player – one in particular, on Chilwell, stood out in the first half – and contested more duels. He had as many shots on target as anyone, and ran further than any Red bar Milner and Fabinho.
Andy Dunn, Mirror
This was another superb contribution from Mane. No wonder he is Lionel Messi’s favourite player. As accolades go, that takes some beating. When the GOAT tips his hat in your direction, you know you are doing something right. Amidst the kerfuffle about lost votes and who plumped for who, it was indisputable that, in the ballot for World Player of the Year, Messi put his cross next to Mane. In the first half, in particular, here, Mane’s contribution to the team in all quarters was immense, his tackling not only impressive in its energy but, generally, sweetly-timed. The same can be said for the other two members of Liverpool’s attacking trio but, considering the amount of miles on the clock this past year or so, Mane’s indefatigability is remarkable. And the lung-testing defensive work here never dulled his tracking sharpness, which is why he was able to gallop on to Milner’s clever first half pass and put Liverpool ahead.
David Lynch, Evening Standard
Mane clocked up his 100th appearance in the Premier League for Liverpool as he completed 90 minutes against Leicester City. And the Senegalese marked the occasion in typical style, grabbing his 50th goal in the division for the Reds as part of another outstanding performance. During RB Salzburg’s visit to Anfield earlier this week, former Chelsea and Arsenal midfielder Cesc Fabregas was compelled to tweet his view that Mane is among the top three players in the world. With Liverpool’s No.10 playing like this and putting up those sorts of numbers, it is hard to disagree.
This story has been reproduced from the media. It does not necessarily represent the position of Liverpool Football Club.