Lat Trick Bot

[Igognito]

Hello all,

a well known problem we have on the server is that many of us are unable to host games.
Recently, an idea come to my mind and I'm planning to explore it.

I'm thinking of making a Lat trick Bot.
The idea is that the Bot will do the Nx(N-1) combinations that we need for everybody to be able to host.

The way I'm planning to do so is by assuming that the Channel is one big game.

Do I know how to do it? Certainly not! I just have the idea for now and know that with some implementation work everything can be done :-D

So what is this post: This is a recruiting post!
For @AHB, @tupac and other PvPGN know hows to join up with me and help!

Cheers
@iL, We also are going to need your help! For starters it is possible that what I want to do might need to alter the code of the PvPGN server. If so I might need your help to set up a developer server and then you might need to upgrade to the new server.

[iL]

The idea is that the Bot will do the Nx(N-1) combinations that we need for everybody to be able to host.

I'd attach it to status, file: http://server.war2.ru/status/status_public4.dat
It updates in real-time when the game being hosted, players join, leave, etc.
So you need Nx*1 combinations and in only a moment when the game is being hosted.
So that would be enough to implement lat trick for only ones who've been hosted the game right last moment. Also you can check "port_isopen" parameter. If "port_isopen":"1", you don't need lat trick at all.

For starters it is possible that what I want to do might need to alter the code of the PvPGN server. If so I might need your help to set up a developer server and then you might need to upgrade to the new server.

Well, i'd prefer to edit and update the code myself, but yes, i think i can share that code to you to let you make changes if you need that.
And yes, i think we can setup a developer server for our modifications.

[Igognito]

The idea is that the Bot will do the Nx(N-1) combinations that we need for everybody to be able to host.

I'd attach it to status, file: http://server.war2.ru/status/status_public4.dat
It updates in real-time when the game being hosted, players join, leave, etc.
So you need Nx*1 combinations and in only a moment when the game is being hosted.
So that would be enough to implement lat trick for only ones who've been hosted the game right last moment. Also you can check "port_isopen" parameter. If "port_isopen":"1", you don't need lat trick at all.

For starters it is possible that what I want to do might need to alter the code of the PvPGN server. If so I might need your help to set up a developer server and then you might need to upgrade to the new server.

Well, i'd prefer to edit and update the code myself, but yes, i think i can share that code to you to let you make changes if you need that.
And yes, i think we can setup a developer server for our modifications.

While it is still very early to discuss such details...
I can tell you that doing lat trick the moment someone hosts wont work.
Until the whole procedure is executed and ports are open (takes a small time for that) people will have already tried to enter and failed.

On the other hand, if we have a flag that tells me who requires the lat trick, that I can certainly use!

What we actually need is that the channel is handled as an open game. Then slowly the channel does the necessary connections.
Some possible problems is that players channel latency might increase a bit.

For those that have noticed, now that I connect by using a 4G network when I join a game I'm shown with red bars. I also create a small lag peek. It takes a few seconds for the red bars and lag to disappear. This occurs because the me x N port opening process that happens automatically when u join a game. This behavior exactly I want to add in channel.

X = excluded usernames (bots, chat only accounts etc...)
N = port_is_open:1 ; port_forced_open:1
M = port_is_open:0

Upon init of bot;
We need M*(M+N) combinations...
and mark this M as port_forced_open:1

When someone leaves the channel we mark: port_forced_open:0
When someone enters the channel we run the init again. Or every now and then (if we see it causes problems to run often)...

Furthermore, we need to keep the connections open or have that port_forced_open expire.

About the code of PvPGN...

The PvPGN is an open project anyway (https://github.com/pvpgn/pvpgn-server) I will only need the version and a dev server.

When we are done, feel free to check my patch if any. Also by the opportunity I might transfer the Elo in \stats. I somewhere read (https://forums.pvpgn.pro/viewtopic.php?id=1782) that in PvPGN we could even have custom fields shown with \stats. That could be the ideal actual as we do not modify at all the normal functionality just add the Custom ladder info.

Anyhow, I will not be starting this before the 20th. I have to much work :/ But I hope until the 20th I will have got the attention of @AHB, @tupac and possible other bot implementors that might help me with this.
Cheers

[iL]

I can tell you that doing lat trick the moment someone hosts wont work.
Until the whole procedure is executed and ports are open (takes a small time for that) people will have already tried to enter and failed.

Why not?
Someone (with closed port) hosts the game, then bot try to send udp 6112 from host to everybody, then everybody will be able to join, won't they?

I'd say, everything you need (if you're the peer with closed ports) is to run that bot on your side. When you host the game, your comp sends 6112 udp to everyone you need, then they be able to join your game. Other players don't need that lat trick bot at all to let them connect you.

What we actually need is that the channel is handled as an open game. Then slowly the channel does the necessary connections.
Some possible problems is that players channel latency might increase a bit.

I didn't understand an idea, the channel as an open game means interchange between each pair of peers? That depends on client (war2), not a pvpgn server.

Also, to keep lat trick working you have to repeat your init UD packets every period of time. So that traffic can be sufficient. I think that will be a big problem, that's why i think about the ways to not require N(N-1) or M(M+N) combinations...

[AHB]

Thanks for your interest @Igognito

I believe port forward free hosting of War2 games is possible as I discussed here http://forum.war2.ru/index.php/topic,4009.0.html

It would not require any modification of PvPGN, it would be an entirely client to client side hack

It is achievable using STUN I believe and has already been implemented in the new Starcraft 2.

I havent spent enough time studying how STUN works to figure it out, but I know it is the key to the puzzle. If someone can put time into this, you can test like this:

1. Study how STUN works
2. Setup a Windows PC behind a public IP and firewall without 6112 forwarded (game host), you'll need to setup VPN or port fwd RDP to get access to Windows
3. Use your Windows PC at secondary public IP and firewall combo without 6112 to try joining the host

From there, on the game host, you would write a program in your language of choice to send some traffic on UDP 6112 over to the person trying to join. From my understanding this would allow them to join your game now, but will probably require further research of how STUN works


After this proof of concept is established and we know it works, here is how we would convert it to production:

1. iL would have to post the IP addresses of every player to the .dat file (currently it doesnt have this). Yeah people might not like that but really who cares, you are giving your IP to everyone when you join a game anyways, the only difference is now you are giving your IP to people you havent joined a game with... this is the only way it is possible to make this trick work

2. The program built for the proof of concept would regularly (every few seconds) check the list of IP addresses of all players, and send traffic to every player UDP 6112 periodically to maintain connections with them.

3. Profit. 23 years of hosting problems fixed forever


(all of the above is basically what iL said, in more words)

I dont know if this is a very scalable solution (lot of traffic as players increase, could overwhelm a consumer router?) but for our small server itd be no problem

I will try to put some time into this. I really think we can do it.

@iL if I solve hosting will you forgive my misdeeds and give me USA-Archer name back lol

[Delete mine too]

I would say unban archer and give him another try.

@Igognito

If I was you i would start reversing war2 when you create a game. Watch all the encrypted traffic, you could make a small program to send and receive UDP traffic. That would emulate the lat trick I think. You would need to explain lat trick well. I also see a human and AI being the fail of this project.

Bot tries to join a game... fails... the bot hosts then everyone joins... not sure exactly how the lat trick is done.

It would be easier having a VPN server. I made one but il firewall was dropping fragmented packets, the fix I heard was MCU at 1500.... haven't had time to confirm that yet. If you would like to try this out I can help you code on the weekends only.

[AHB]

Like @tupac said, good point, might be less work to build VPN as part of the war2 client I bet, that might be another hack around

thanks for the support, I am the longest ban in server history FREE ARCHER 2018 served 3 years for my crimes let me out with community service lol @iL

[AHB]

It is not so simple as sending traffic outbound from client A to all other players on 6112 in order to open 6112 to them:

- NAT does not map outbound connection on 6112 to be public-ip:6112, (otherwise only one person would be able to use web, email etc at a time)
- Instead it chooses a high number port like public-ip:49123 to map to private-ip:6112
- In order to join player A's game, player B's war2 client would have to connect to player A:49132 instead of A:6112 (not sure if this will ever be possible because would require a change to the war2 game itself right?)


this is a good resource:
http://www.brynosaurus.com/pub/net/p2pnat/

[AHB]

A good, simple description of UDP hole punching, I added notes on how we could implement it for war2.ru:


1. User A and User B establish a TCP connection to PvPGN server. Once established, both users send to PvPGN server a port on which they can be contacted. We'll call User A's "Port X" and User B's "Port Y". (this would be a Python program or whatever that gets added to War2Combat and posts public IP and port number to a war2.ru webpage, which collects it and adds to list of user/port combination to dat file)
2. PvPGN server shares the port information of User A with User B, and vice-versa. (iL outputs IP address and port number of all players in JSON format to .dat file, which is then downloaded by Python program embedded in War2Combat)
3. User A sends a UDP packet from Port X to User B on Port Y. User B's firewall will of course reject this packet because it was not expecting it
4. User B sends a UDP packet from Port Y to User A on Port X. User A's firewall will forward this port internally to User A, because User A's firewall just saw a packet leave User A through it out Port X and to User B on Port Y (and because it's UDP, it has no idea that it was rejected), so it assumed that a packet coming from User B on Port Y to User A on Port X is a response packet, so it lets it through.
5. User A sends another UDP packet from Port X to User B on Port Y, and it is also allowed through User B's firewall for the same reason.
6. Users A and B can now send UDP packets back and forth, and Control's participation is no longer needed.

The only thing I dont understand is how would war2 know to connect to a game on port 49000 for example instead of 6112/ is it possible?




Source: https://serverfault.com/questions/425258/does-hole-punching-require-a-udp-rendezvous-server-or-could-it-be-a-tcp-one?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa

[Igognito]

@tupac, @AHB, @iL great that I have everyone in this!

and thank you for the brain storming...
I intent on looking to this problem, but I do have limited time my self.

To be honest, I do believe the best is to solve it on the PvPGN server level.
The server knows all ips, all UDP ports.
Also some solutions I have seen from other gaming software is implemented on chat channels.
Example: Voobly
Before the launch a game the open a small chat channel. There all users try to connect with all users (this takes some time) when all connections has established the launch the game.
This behavior I think is what we refer to as the LAT trick. Also war2 does exactly that when u enter in a game.

so the ideal would be that the client can send a UDP packet to a specific ip address.

ps:stopped suddenly will continue

[AHB]

It fundamentally cant be solved at the PvPGN level, actually there is no need to modify pvpgn at all I think. There needs to be a server which receives the port mappings for each client and distributes them to all other clients, however this would be much easier written in PHP with a JSON HTTP output rather than hacking with C++ TCP connections in PvPGN.

On the client side I would prefer Python to make HTTP posts, but whatever will work, eventually that will be compiled into an exe, and added to the War2Combat startup script to run whenever war2 is launched

Here is another good source:

https://steamcommunity.com/app/427520/discussions/4/371919771755670102/


At this point I understand how NAT hole punching works, I'm just not sure how we will get the other war2 clients know to connect to different high number ports rather than 6112 depending on who is hosting?

Currently we know people can change their War2 port, so I guess I just need to understand how the client knows to connect to a host's custom port rather than the default 6112.

[Delete mine too]

It fundamentally cant be solved at the PvPGN level, actually there is no need to modify pvpgn at all I think. There needs to be a server which receives the port mappings for each client and distributes them to all other clients, however this would be much easier written in PHP with a JSON HTTP output rather than hacking with C++ TCP connections in PvPGN.

On the client side I would prefer Python to make HTTP posts, but whatever will work, eventually that will be compiled into an exe, and added to the War2Combat startup script to run whenever war2 is launched

Here is another good source:

https://steamcommunity.com/app/427520/discussions/4/371919771755670102/


At this point I understand how NAT hole punching works, I'm just not sure how we will get the other war2 clients know to connect to different high number ports rather than 6112 depending on who is hosting?

Currently we know people can change their War2 port, so I guess I just need to understand how the client knows to connect to a host's custom port rather than the default 6112.

The anti hack il made could handle that.

[iL]

@iL if I solve hosting will you forgive my misdeeds and give me USA-Archer name back lol

That looks schizophrenic, you hack/ddos the stuff by one hand and make some useful things by another at the same time. It happens every time.
I have no idea about your reasons, but honestly, i'd prefer you to keep far from our server and things.

To be honest, I do believe the best is to solve it on the PvPGN server level.
The server knows all ips, all UDP ports.

Well, if you think to release that on pvpgn - ok, but i don't think i can help you with that pvpgn research. Of course i can integrate your code into server pvpgn or share the current pvpgn source code to you.
But i have no idea to resolve that on pvpgn server level.

But you right, pvpgn knows all the ips and UDP ports.
And: http://server.war2.ru/status/status_public4.dat is a public part. Or course status has private part that also knows all the ips and ports (gets from pvpgn).
So we can share that info to your utility somehow.

So, I think we have different ways to resolve ips and ports. But i see another problem:
Can you firstly make another part? The heart of the project. A program/function to UDP request to the required ip/port from war2 port?
Something like:
int udp_request(char *ip, int port)
{
...
}
I mean, some code calls your function, your function resolves your local war2 outgoing UDP port somehow (maybe get it as a parameter also), then send UDP where required and then you will be able to receive incoming UDP packets to that ip and port.
That's a small step, but would be the main part for the project!

Can be easily tested with you, me and several friends.
So, the test will be easy: i close my ports, you and 2-3 your friends tell me your ips, i host the game, and begin to send UDP to your ips, you try to join.
I'm not sure if you can use UDP port captured by another program (war2), so here's the first problem i see here. Maybe we have to inject that UDP info war2 executable code to let it send packets when that port is already opened for reading/writing by war2? Or we can open UDP on the same interface/same port from another process?

After that part will work perfect, we can analyze traffic and find the way to get all (or all the required) ips and ports.

[Igognito]

There are problems with having bots doing this.
If for each user we need a bot that does the ping it is a problem.

The reason I want to do this at the server level is so that we do not require double logins for each user.
Probably the easiest is to have a bot for each user that knows the ips and does something like this:

for all IP in IPs
  echo "Lat Trick" > /dev/udp/IP/6112
end for

and

netcat -ul 6112

This is something we can try among us. If by both of us doing this the ports open then we have verified that all we need is to send/receive a package...

I was thinking of doing it like that:
The server send a packet to the clients that has encrypted the IP and UDP port of the client to send a package back
then a utility decrypts the ip and udp port and send back a package...

There we are everything is fixed... we just ping pong each other over UDP

now, some programmers could probably listen their outgoing packages and find some ips... but I do not think that is too much of a problem...

worst case scenario someone will use that information for a DOS attack. An average firewall should be able to handle that.
What is dangerous is if the can code inject something to that ip. But I think they shouldn't be able

Full design:

Introduce \lattrick chat command
\lattrick sends to user IP a UDP packet at a port that a custom app listens
the app when it receives the packet starts to listen for encrypted ips
the ping pong starts
when all ips finish the server sends the confirmation message.

the code should be similar with the \ping command code
we just have the pong part going to the ip stated...

[AHB]

@Igognito

Your design:
Although it is one solution, the flaw that I see in your design is it requires user interaction each time before hosting a game. How would the server even know who you want to open ports with? You would have /lattrick <username>? And then the host would have to repeat for each player that he wants to be able to join?

My design:
- No user interaction required, no chat bot commands, runs in background and opens connections to every player so that anyone can host

1. Client A wants to host a game, but is behind NAT with no port forward
2. Client A downloads a list of IP addresses and ports of all other players of the server periodically by downloading HTTP JSON response from war2.ru. Keep in mind this is not a "bot" in the sense of a chat bot, it is a program bundled into War2Combat that runs continuously in the background without user interaction or even knowing about it
3. Client A periodically sends UDP traffic to every single one of those IPs/ports from port 6112, however, his NAT router translates 6112 to a high number port like 49123. The first time traffic is sent, it will be ignored, until the otherside initiates a response (since it is doing the same periodic sending of traffic), after that the two will be able to communicate freely
4. All of those clients that Client A is sending periodic traffic to can now respond on port 49123, and join games on that port, however, they need to know that is the port Client A is listening on so they have to get that info from PvPGN, who is already maintaining this information. (We might need to have a way to update PvPGN with this NAT port, which tells the War2 client which port to join a game on, this is the only step I am unsure how to code)

There is no /lattrick command needed, and definitely no point in encrypting anything. Once you are in a game, you know the IPs of all players right now anyways, it is not secret information, the only difference now is that you will know the IPs of people you arent in a game with as well

Since Client A is sending periodic UDP traffic to every single other player on the server, anyone can join his game without having to run a command, send an IP etc

UDP hole punching example in Python:
https://github.com/dwoz/python-nat-hole-punching

2 clients connect to the server, the server tells each client about the other client's IP and port, then the clients talk to each other over the IP/port