Making his Premier League 2 debut, Koumetio played the full 90 minutes alongside Sepp van den Berg at the heart of the Reds' defence at Rush Green FC.
Luis Longstaff, Tom Clayton and Liam Millar – the latter from the penalty spot – were also among the goals for Liverpool.
Barry Lewtas welcomed back Leighton Clarkson after a three-game suspension, while Glatzel got more valuable minutes when he appeared as a 67th-minute substitute.
Before kick-off, both sets of players observed a minute's silence ahead of Remembrance Sunday and displayed poppies on their shirts.
Lewtas' side made the perfect start and took the lead inside seven minutes when Longstaff ran onto a through ball from Clarkson and kept his cool to finish.
The home team almost levelled on 17 minutes as Keenan Forson found space in the penalty area, but his effort was superbly kept out by Caoimhin Kelleher.
However, the young Hammers did level with 22 minutes on the clock.
A charging run down the left flank by Manny Longelo saw him get beyond two Liverpool defenders and his cross was turned into the net from close range by Oladapo Afolayan.
West Ham then went in front 10 minutes before half-time, Mesaque Dju putting the finishing touches to a three-man team move involving Afolayan and Kamarai Swyer.
Lewtas' charges had a glorious chance to equalise just before the break when captain Millar drove into the penalty area, only to see his close-range effort saved by Joseph Anang.
Millar was causing a lot of problems for the Hammers' backline and in first-half stoppage time another driving run down the left saw him find Longstaff in space, though the No.9's strike was brilliantly kept out by Anang.
Within four minutes of the restart, though, the Reds were level when Clayton pounced for the second week running – this time netting from close range following a corner.
In the 56th minute, the hosts were reduced to 10 men when goalkeeper Anang conceded a penalty for bringing down Longstaff as he went through on goal.
Millar stepped up to take spot-kick duties and duly sent replacement stopper Daniel Jinadu the wrong way to make it 3-2.
Both teams continued to create good chances and Kelleher and Jinadu were both called into action on numerous occasions.
However, with West Ham looking for an equaliser, Liverpool exploited space in stoppage time and Glatzel put the game beyond doubt with a superb finish.
LFCTV GO: Watch the best of the 4-2 win
Reflecting on the win, Lewtas told Liverpoolfc.com: "We started really well and it was end to end and we created a lot of chances.
"We played some good stuff in the second half, coming back from 2-1 down and we pulled our sleeves up and got the equaliser.
"Despite them going down to 10 men, West Ham were still a threat. They were direct and physical and it was a tough game.
"It was great to have Billy back. He's a really good kid and he fills the room with his personality and his smile. I was really pleased with him because in the first half he had some real tough moments but in the second half he showed the character that he is.
"We were really pleased to have him with us and he had a really good centre-back partner in Sepp, who led the backline ever so well for us, like he has done in the last few weeks. Sepp really helped Billy to settle, which was great.
"It was great to see Paul get a goal at the end. He has looked really sharp in training. He holds the ball up really well for us, he brings others into play. He had one shot blocked and scored from the other.
"We are being a bit careful with Paul after his injury but that doesn't mean we are wrapping him up in cotton wool. We are just easing him in until he's fully ready to go."
Liverpool U23s: Kelleher, Sharif (Savage, 78), Beck, Van den Berg, Koumetio, Clayton, Bearne (Glatzel, 67), Clarkson, Longstaff (O'Rourke, 89), Cain, Millar.
Unused substitutes: Winterbottom, Ritaccio.