Server Lag : what it is and isn't....

[tk[as]]

i got impatient and looked into it myself... i think i was kinda on the right track:

Skype made a technique popular that is called firewall hole punching: A server with a public address is contacted by the clients. Then it sends an answer back to client A telling it to send udp packages to the router of client B on a port it expect the router of client B to use as source port for the next outbound packet. And it tells client B to send a packet to the router to client A on a port it expects the router of A to use as source port for its next outbound packet. If the prediction is correct, A and B can now talk directly which each other.

[iL]

One way to fix this *COULD* be to change the route by using a fast proxy or VPN. A VPN establishes a connection over HTTPS between the client and the VPN provider, from where the data would most likely be routed through a different set of servers to the other players.

Interesting if that's posible to make something like NAT substitution for such problem clients. I mean some kind of wrapper between war2 client and network adatper. Say, the problem is between 1.1.1.1 and 2.2.2.2. War2 from 1.1.1.1 sends UDP to 2.2.2.2, that wrapper translates 2.2.2.2 to 3.3.3.3 (another client which connections to 1.1.1.1 and 2.2.2.2 are faster) like kind of vpn. 3.3.3.3 translates traffic back to 1.1.1.1 after receiving answer from 2.2.2.2.
Such kind of distributed UDP antilag-network. With dynamic scanning network performance between each pair of clients and finding the best routes.
Such solutions should be implemented somewhere already. Would be great to get such opensource solution and use for our tasks.

as im reading ... would it be possible to use the lat trick for multiple players?

Also good question. I'm pretty sure it's possible with some kind of software (like loader), every client just sends UDP packet to any other clients.

And yes, some new and popular software products (like skype) have such solutions integrated into it.
Appreciate if someone can offer some libraries for keeping UDP interchange between clients or w/e to integrate into C++ project.

[tk[as]]

http://gamedev.stackexchange.com/questions/9971/networking-without-port-forwarding

This is where I found out the firewall hole punch trick

[tk[as]]

Based on the way it was explained, I would imagine the lat trick would work for multiple people without additional software..

4 players. A B C D

A hosts. B c d try to join (all sending upd packets to A) .. none join.

B Hosts. A c d try to join, only A would get in because at this point player A and B have both sent udp packets to eachother.

C hosts and all try to join. this time both player A and B are able to join.. D is left out because he has only sent udp packets, nobody has tried to send him anybyet.

Once D hosts.. all should be able to join.

Long process, but should work, no?

[tk[as]]

The only reason I can see it wouldn't work is if player D (or maybe C also) has disregarded packets  being sent to it from player A since there were a series of packets being sent previously, with no immediate response back

[Lambchops]

would it be possible to use the lat trick for multiple players? i've only ever tried in 1v1 .. But theoretically if ...........

...... you're talking about our ping in relation to the server.war2.ru.. what about /ping (specific player)   .. the number shot back at me is the time it takes in milliseconds for our computers to communicate with eachother, correct?  so, a player with a ping of 3500 vs 350 is going to make a noticable difference in game.....   

so for the bonus question .. .....the only things i can come up with is somehow form a TCP connection (even if only temporarily) with all of the players on the server so you auto-send/recieve UDP traffic to those players after that   .. or somehow, when you first log onto the server and successfully send/receive UDP packets to the server, have the server forward your UDP packets to all other players on the server so there is a "connection"........

Cool.    Can't quite award full points for the bonus question, but I'm loving the way you're thinking. Definately asking the right questions. I'll write a post with my best answers to this next, But first, here's the answer I wrote yesterday.

... and everyone please note: This is all just theoretical. It seems to make sense to me, but I'm very happy for anyone to explain it to me correctly if any of this is wrong, that's why I started this topic.

....and for the extra bonus points... WHO WAS PAYING ATTENTION? - What small change could have been made to the battle.net protocol to eliminate the whole "opening ports" to host a game issue?

OK then, what do we know about this whole port forwarding lark? Sure, by rights you should have port 6112 (or another if you have configured a change) forwarded to your gaming computer's local address, so that the router knows where to send this traffic. Fine.

But we also know its pretty easy to get around, because even without this you can still join a game and have your client happily sending and receiving UDP packets to and from up to 7 other computers simultaneously. You can even host by using the “lat trick” to get around this.

Although I haven’t actually spelled it out, the reason you can join games is actually exactly the same mechanism as the “lat trick”. By virtue of everyone’s IP address and port being circulated to everyone else by the host, then everyone sending everyone else a UDP packet (and no doubt a few re-tries) over their game port, in effect the “lat trick” is just being done for you automatically for all players in the game.

So why can't you host then?

The problem is with that very first "I want to join" message that your client sends to the game host.

This goes back to the evolution of the game: the multiplayer game was designed to operate on a local network. The battle.net part, despite the nicely integrated graphics etc., is very much a completely different thing onto which they “bolted on” the existing multiplayer game like an after-market carburettor. The game list is little more than a very simple ad for the host's IP address and port, there isn't really any help beyond that. It's up to your client to try to connect to the host, but until it does the host doesn't know that the client is trying to connect.

This is the big difference between hosting and joining, when you join you have already received the host's address from the server and sent outbound traffic when you asked the host if you could join. As mentioned, the host circulates everyone’s addresses so they can all send a few outbound packets to each other to initiate the transaction, and its all good.

So in this entire scheme there is only one point at which a UDP transaction is being attempted where one of the clients is 'blind', .i.e. it hasn't been pre-supplied with the other client's address and port. That is when someone first tries to join a game. The joining client obviously knows the host's address, but the host has no clue. Why? Maybe it was by design to reduce server load on the original battle.net servers, idk, but both clients already have an established TCP connection to the server. So why not?

All that needs to happen is: when a client attempts to join a game, it notifies the server (of course over the TCP), and the server in turn notifies the host of the joining client's address and port, which is 6 lousy bytes! (plus standard transport overhead). Then the host just has to send a few UDP packets out to the joining client. That's all.

The packets can be anything. Null packets would probably work fine, whatever, as long as it doesn't crash the client with garbage, its all good. We have then done our “lat trick” between the host and the joining client which is the only one not already being done. End of hosting issues forever. GG.

 
Want proof that your clients are actually doing the “lat trick” for you when you join a game? Try this:
(I've never tried it, but it should work;)  /me quietly prays that it works...

When a game is over, but before you leave that game, pick somebody there who can't host, (but has a good connection) to host the next game. Then decide on a nice easy game name, like “zz” or something. Then everyone leave at the same time. Have the next host (who can't host) immediately host the game, and everyone else immediately type in “zz” and join. Don't worry about a password, anybody who wasn't in the previous game won't be able to join anyway.

Everyone should be able to join the next game, because the “lat trick” won't have timed out from the previous game. In fact this should work from the pre-game lobby, without even starting the game (maybe all change race before you leave to be sure). I can't see why 4 players, none of whom can host shouldn't be able to decide on a host and a name then, for instance, all join the EF_gamebot game, then leave and have someone successfully host a game. *Of course all players would have to be in the gamebot lobby at the same time, you can't all join and leave one at a time...

[tk[as]]

When I initially answered I did not realize the server could request player A to send a udp packet to player B and visa versa .. I did see a flaw in the answer I gave  (or what I believed to be a flaw) .. if the server were to try to simply forward UDP packets from player A to player B, player B would see those packets being sent from the servers ip address, not player A's ... unless there was some kind of mechanism that was able to trick player B into believing the packets were directly sent from player A and visa versa

[Lambchops]

.. you're talking about our ping in relation to the server.war2.ru.. what about /ping (specific player)   .. the number shot back at me is the time it takes in milliseconds for our computers to communicate with eachother, correct?  so, a player with a ping of 3500 vs 350 is going to make a noticable difference in game

My understanding of the PvPGN '/ping' command is this: It periodically pings each connected client and keeps a record of the result. In the channel /p will return the last recorded ping time for yourself, in a game it will list the last recorded time for each player in the game, and /p (player) returns the last recorded time for that player. They are all accessing the same info which is the roundtrip -server--> player--> server- time.

The server can't give you a ping time between two clients. The ping has to come from one of the endpoints you are testing (i.e. your computer). Any line starting with '/' is sent to the server, so for this to happen the server would have to send back a request to your wc2 asking it to ping another player then display the results. PvPGN is a generic battle.net emulator and not in any way specific to wc2, and this type of ping display isn't built into wc2, so without mods to the client this can't happen.

But they way you're thinking is right, its all about the connections between each client. Where you DO see this is the 1G --> 6R bars next to each player in the pre-game lobby. But in reality the raw ping time is not the most important factor; you rarely see anyone with anything more that a 3y ping managing to join a game, and they can be fine with regards to lag. What you do see is people's connections going 6R periodically etc. A stable connection.... consistant speeds and no packet loss is much more important than raw speed. Most connections these days are fast enough for wc2, although YES any game where 2 clients are pinging 3500 will have unplayable lag.

(edit)
oh and of course you can ping another player any time you like, as long as you know their IP address. If you are chatting with a friend they can message you their address i.e. 111.222.33.44 and you can just type "ping 111.222.33.44" at a cmd prompt.

.... you can easily find out your external IP address, just google "my ip"

[Lambchops]

When I initially answered I did not realize the server could request player A to send a udp packet to player B and visa versa ..

No, it can't... thats the point. It would have taken 5 minutes and a few lines of code to add when they were working on the wc2 source tho... it would take a lot longer for someone to reverse engineer and patch it in now, but its possible.

[tk[as]]

I realize it cant as of right now.. but didn't realize until yesterday it even has the ability to if given the proper instructions (code)

Interesting stuff though.

[Lambchops]

i got impatient and looked into it myself... i think i was kinda on the right track:

Skype made a technique popular that is called firewall hole punching....

Nice one. Thanks for that, I didn't know that term.

A server with a public address is contacted by the clients. Then it sends an answer back to client A telling it to send udp packages to the router of client B on a port it expect the router of client B to use as source port for the next outbound packet. And it tells client B to send a packet to the router to client A on a port it expects the router of A to use as source port for its next outbound packet. If the prediction is correct, A and B can now talk directly which each other.

.. and yes this is exactly what the server should be doing, but it isn't.... the host client is doing this, but only once the guest clients have successfully connected to the host, which is why the problem manifests at this point.

... the server is doing half if this: its giving the joining client the address of the host client, but it isnt giving the host client the address of the joining client.... which is what the 'solution' I posted suggests.

[Lambchops]

Based on the way it was explained, I would imagine the lat trick would work for multiple people without additional software.. 4 players. A B C D ......./.......Long process, but should work, no?

That's what I think yes. The thing to remember is the only piece of the puzzle missing is the host sending packets out to the joining clients, so if you wanted to do the lat trick with 4 people, and didn't have another game to join, easiest way would be to decide on the host, then have the other 3 people make games. First the host tries to join all 3 games in succession then makes his game, then the other 3 should be able to join that game.

The only thing you are fighting is, whatever internal timeouts the host has for remembering connections. It would depend on your router firmware. I don't know how long this is, but it cant remember the IP address of everything you've ever connected to for ever....

[I hate naggers]

Want proof that your clients are actually doing the “lat trick” for you when you join a game? Try this:
(I've never tried it, but it should work;)  /me quietly prays that it works...

When a game is over, but before you leave that game, pick somebody there who can't host, (but has a good connection) to host the next game. Then decide on a nice easy game name, like “zz” or something. Then everyone leave at the same time. Have the next host (who can't host) immediately host the game, and everyone else immediately type in “zz” and join. Don't worry about a password, anybody who wasn't in the previous game won't be able to join anyway.

Yes, the players would be able to join that game, but the non-host players would 'conflict' with each other

[Lambchops]

One way to fix this *COULD* be to change the route by using a fast proxy or VPN. A VPN establishes a connection over HTTPS between the client and the VPN provider, from where the data would most likely be routed through a different set of servers to the other players.

Interesting if that's posible to make something like NAT substitution for such problem clients. I mean some kind of wrapper between war2 client and network adatper. Say, the problem is between 1.1.1.1 and 2.2.2.2. War2 from 1.1.1.1 sends UDP to 2.2.2.2, that wrapper translates 2.2.2.2 to 3.3.3.3 (another client which connections to 1.1.1.1 and 2.2.2.2 are faster) like kind of vpn. 3.3.3.3 translates traffic back to 1.1.1.1 after receiving answer from 2.2.2.2.
Such kind of distributed UDP antilag-network. With dynamic scanning network performance between each pair of clients and finding the best routes.
Such solutions should be implemented somewhere already. Would be great to get such opensource solution and use for our tasks.

Yes this would be great, but it would not be a small project. Say you built in a layer (storm.dll somewhere, I guess?) that detected packets with an extra custom header designating that they should be redirected elsewhere then had it just strip that header and re-transmit the packet. Then you would have to have each client test its connection to each other client via a third and decide on the best solution. For an 8 player game this means each of the 8 players doing 36 individual load tests. You would have to decide on thresholds for proxying and repeat the test because if everyone decided to use a single client with a nice connection to proxy everything, that client would suddenly have 49x the nework load and its performance would drop..... it's a "movable feast" as my father would say... but it would be very nice once implimented.


Oh, and the gold standard would be this happening dynamically, in game... like when a player has an intermittant problem connecting to some players, it would detect this in real time and start proxying through another client.....  Hey, you code it ... I'll cheer you on

--------------------------

In effect it's a kind of VPN ... without the P.

 But i think (not sure)  VPN's all work through a server, not peer-to-peer, don't they? I'm not aware of any open source project that does this, but would love to hear about it if there is.

---------------
(edit) ..... (again lol)

no, by the looks of it you can configure them to do all sorts of stuff...
check this out maybe.
... very late here, I need to get some sleep.

[Lambchops]

Want proof that your clients are actually doing the “lat trick” for you when you join a game? Try this:
(I've never tried it, but it should work;)  /me quietly prays that it works...

When a game is over, but before you leave that game, pick somebody there who can't host, (but has a good connection) to host the next game. Then decide on a nice easy game name, like “zz” or something. Then everyone leave at the same time. Have the next host (who can't host) immediately host the game, and everyone else immediately type in “zz” and join. Don't worry about a password, anybody who wasn't in the previous game won't be able to join anyway.

Yes, the players would be able to join that game, but the non-host players would 'conflict' with each other

Really? I can't see why. Have you tried this?