As i see, a C++ class/library to include into war2 loader.
Just thouhgt: would be great to find good network programmer/engineer for war2 project:
We could handle the whole war2 network behavior ourselves: to guarantee all the UDP connections between all the cilents, to manage and find optimal routes for that connections
Sounds like fun, but as I say, a lot of work. Personally I have far too much to do right now to be able to seriously commit to a project like this, although if you want to do it, I will help where I can. Even just writing these posts is taking more time than I really should be spending on gaming.
There are a lot of things to consider when routing network flow. Here is an example of some of the basic issues:
[Network Flow I] …. after that you can start factoring in some of the more technical bits.
Perhaps there is already an open source solution that would fit what we need for wc2, which could save the need to code the flow analysis/decision making from scratch, but even assuming that is is already done, just hooking this in along with the necessary support code to adapt the client to use it is a big job.
We must also remember that what we are always striving for is reliability and minimum latency, not maximum flow. Every extra step that is placed into the nework logic inheranty adds latency. A small amount of this to side-step disasterous connections between individual clients could add up to a net gain for the reliability of individual games, but too much of it and you're using a flame-thrower to get the weeds out of your lawn… the weeds are gone, but so is the lawn.
reroute if clients will suddenly change their IP during the game.
I think this would be a very rare event.
UDP over internet is far from perfect and has lots problems. So, some kind of layer between raw war2 UDP and internet (like distributed UDP network between war2 clients) based on new technologies and implementation is what we need.
UDP packects are inherantly no less likely to arrive at their destination than any other type. By rights, it *SHOULD* make absolutely no difference. There is a difference, but that is not due to any problem with the UDP spec, it does exactly what it was designed for. An explaination requires a bit of discussion about netowork data streams, so....... Packets are arranged something like this like this:
This is a very simplified version just to illustrate the principles, there are other fields not listed hereInternet Layer: An IP packet:
{version, size, protocol, src address, dst address, {Payload - 'size' bytes of data}}
Transport Layer:
UDP Packet:
{src port, dst port, size, checksum, {Payload - 'size' bytes of data}}
TCP Packet:
{src port, dst port, sequence number, size, checksum, {Payload - 'size' bytes of data}}
So at the internet layer, a UDP packet looks something like this:
{version, size1, 0x11, src address, dst address, {src port, dst port, size2, checksum, {Payload - 'size2' bytes of data}}}
Here
size1 is the size of the entire UDP packet, which is the payload for the IP packet, and
size2 is the size of the data which is the payload for the UDP packet.
... and a TCP packet would be something like:
{version, size1, 0x06, src address, dst address, {src port, dst port, X , size2, checksum, {Payload - 'size2' bytes of data}}}
So accoring to the original design, the IP packets are only being routed based on the information in the IP header which is pretty much just the address and the number of bytes. Its only when the packet is received by the network drivers at the other end that the IP headers are stripped and the payload is then passed on to the appropriate transport layer driver that the internal headers are inspected.
The IP header does, however contain a
protocol field, which for these 2 cases will be set to either 0x06 for TCP or 0x11 for UDP, so servers along the way can see what type of data is in the payload and possibly
decide to do something different with the packet.
Without an actual decision being made about it somewhere along the way, neither TCP, UDP or any other transport layer protocol is any more likely to be reliable than another. The main difference between TCP and UDP packets is the inclusion of the sequence number in TCP and the behaviour of the drivers handling transaction. TCP simply puts a number on each packet when it is sent, like the page number on a book.
So if the TCP driver receives packets 1,2,3,4,6,7,8 it sends back a request saying "hey I'm missing packet 5 - send that again!",
and it blocks forwarding packets 6,7,8… etc. to the application until packet 5 is received and inserted into its correct spot in the data stream. This great for reliable data transfer, but terrible for anything that requires real-time transfer. This is why things like VoIP and live video streaming use UDP, because for them having the latest real-time data is more important than worrying about a tiny fraction of the data that was not received.
BUT…. and this is the big but… what if you are attemting to write firmware for a network router? What do you do when there is congestion in the network and you simply cannot transmit all of the data that you have received? Lets pretend that you are only getting TCP and UDP packets (indeed they make up the majority of the transport layer bandwidth). What if you drop a TCP packet? The driver at the other end is just going to send back a request for the missing packet, then the source is going to re-transmit the packet and you have just doubled your congestion problem.
On the other hand, most of the UDP traffic is stuff like VoIP and you know that
a) losing 1 in every N packets in a VoIP stream doesn't stop it from functioning, it just gives a corresponding loss in signal quality and more importantly
b) If you drop a UDP packet, most times this will just be accepted by the clients and you won't have to handle the traffic from them requesting and retransmitting the lost packet.
Considering this, its no surprise that generally when a server is facing network conjestion issues the first thing they start doing is dropping UDP packets.O.K. so lets just stamp all our packets with a 0x06 instead of an 0x11 right? So the servers will think the payload is TCP and be less likely to drop it… Nice in theory but there's a couple of issues with this. Firstly, the IP packet header is being written at the driver level, so it's not just a matter of altering wc2 client you would have to be hacking the network services on the machine it's running on (no thanks) and second, stateful and application layer firewalls are quite commonplace these days. These firewalls
do inspect the payloads of internet, transport and even application layer packets looking for things like malicious activeX controls and the like. These would instantly flag incorrectly labeled IP packets and a the very least delay them if not block them all together.
ISPs also use a certain degree of this type of packet inspection to manage their bandwidth, for instance on Monday afternoons my torrenting speeds drop to about 10% of what I get the rest of the week. Torrents are distributed via anonymous peer-to-peer connections, and in theory my ISP should have not even know that the packets contain torrent data, but you can be very sure that they do.
These types of issues are why I suggested using a fast VPN *COULD* be a way to improve a bad connection. Inherantly VPNs have a lot more overheads associated with them so should have higher latency that just sending UDP packets, however govornments, corporations and all sorts of
high paying customers for the ISPs tend to use VPNs, and being secure, there is no way for anyone to have a look inside and know exactly whats there, as a result many ISPs tend to give high priority to this type of traffic, and they are the last type of packet that is likely to be vonluntarily dropped by a server.
…. and at this point, like so much that happens in our little community, our issues start to become less about network engineering and more about social engineering.
