If he's been playing for hours maybe he's still playing on the old patch and the server hasn't been updated. As soon as the server update process starts on that server he'll get booted and won't be able to join unless he updates himself. I didn't realise you could manually connect thru the IP address?
Yeah I would open the steam browser then click my name tab and go to friends, when I would join his game from that screen it would give me his servers ip in a box and i could net.connect from the console
net.connect 123.123.123.123:12345 an example of the console command you use to connect manually