P2P technology in bzflag

[1veedo]

I think this would be very beneficial because servers would have the capacity to host more players. On the downside, cheating might be easier, and depending on the node relations, some people may have lags much higher while others lower. However, each dedicated node could analyze data itself, and if the server realizes that one didnt report something the other did, it would be obvious that soembody was cheating.

We could just have a couple "dedicated" nodes on the network that everybody else went to (probably people w/ lower connections or who turn it off).

A dedicated player would be one that is on a while, has the feature enabled, and has the bandwidth. It would collect information sent by whoever it was "supporting" or hosting then send that data to every other dedicated host as well, and provide feedback to the main server on cheating, etc. The server itself would be a dedicated player, obviously. The dedicated hosts would then upload the information gathered from other dedicated nodes to its players.

Here's a simple diagram, not pure P2P, but it has aptitude none the less.
<<Info Dedicated <-> Players
Server <-> Players || info
<<Info Dedicated <-> Players

This would tremendously increase the capacity of one server. There could even be secondary nodes, that would get info from a dedicated then send it all to maybe just a couple of people. It is possible that lag for the dial-up's and players w/o the feature established to experience slightly larger lag times, but not by much.

There would still be a limit though, based on the server's and dedicated's bandwidth. If there were 5 dedicated players, that would require ~3 * 5K * 8 = 120kb to send all the information about it's current players to the others. 120kb isn't _too_ much, but then you have to consider that each dedicated has to still send that information, and then some, to all its players. Staking to 5K in the end, to each of its maybe 4 players = 120 + 40 * 4 = 280. (to send to each node would be less sense the weight is distributed and thus only the info about its "hosted' players would need to be sent)

Then there's the problem with a player disconnecting, the supported would have to find another place in the network. Suddenly, 8 players are going to have huge lag stats. And those 8 are also going to experience lag w/ everybody else! If we set up some sort of auto-max players, based on bandwidth per host, they might have to be kicked or temporarily placed in low priority observers until somebody can provide upstream to them.

Pros
Actually reduces cheating
Increases server capacity

Cons
Dedicated nodes suddenly cutting off

Indifferent
True P2P probably wouldn't work; there is going to be a limit based on the number of dedicated and their bandwidth. If one dedicated has 180 and there are 5 others, everybody supported by it would have bad lag.

Something like
(bandwidth - num of dedicated * 3K?? * 8 ) / (5 * 8) = number of users able to be supported by node. As you can see, we'd get a sortof hyperbole goign on where the more dedicated nodes, the less each node can provide for.

Just an idea, I think it has potential.

[JeffM]

Generaly the P2P networks are not setup for the transfer of changing realtime data, You'd have to maintain this common gamestate across all notes.. and that sync would take bandwith and time. You would actualy be adding MORE latency as the servers would not allways have the most up to date info. In a file sharing system yes you get more transfer rate. But they do it by spreading out the data and sending from more then one host. That works because all those hosts have the same data. Here the data is allways changing ( the game state ), so you can't realy just get bits and pieces from difreent servers, they may not all be in sync, and if you wait for them to sync then you've just blown the validity of the data since it's changed.

In the game industry the only types of disributed server networks that work, are ones where the servers are interconected via a very high speed backbone ( usualy a lan ), and the spreading out of the load is for CPU load, not for bandwith reasons. This is most commonly done for the Persistant world Masivly multiplayer online RPG games, where you have diferent machiens service the various sections of the world, beause there are just SO many players. Those systems are still feed by a number of pipes from the net, but they all feed into a single lan that the servers use to comunicate. The speed of the lan is much higher then the internet, so the servers can sync faster then the net and it's all good.

p2p networks are great for uses where latency isn't important, and the data is somewhat constant. like file transfers. Blizzard is using a bittornet type mechanism to distribute it's patches, and it's one of the most unstable systems they have. most people are behind firewals, and it just dosn't work right. Now the ONLY use this for a file transfer system not for game stata data.

Your bandwidth fomula is only viable for bytes transferd, not for latency. in a game, it's not realy the amount of data that you can send ( to a point ), it's how fast you can get it there, since the data is invalidated allready right after you send it ( the tank moved ).

[RPG]

I was thinking the same thing as Jeff before I saw his post. If the data was sent out from the "master server" to all of the nodes, then to all the players, the length the data would have to travel could be longer than if the node didn't exist. (this is if the data takes the most direct route to the destinatinon)

(On my little diagram, the dotted line is without the node and the solid line is with the node)

Please help me if I didn't understand the idea correctly.

[1veedo]

That's sortof the idea, I guess :-/

I was thinking of having each player send his data to everybody else on the server, but then realized that cheaters would love that

[RPG]

Yea, I pondered the idea of a more efficent BZFlag awhile ago, and one of my ideas were having each client send their data to all other clients. Boy, that would make a hell of a laghouse. It's serverless BZFlag. I think they have it running pretty efficently with UDP now.

[JeffM]

rpg, what you describe is a star or round robbin method. They have there uses for some things. but the most proven solution for high speed realtime gameplay is the client/ server model. It has the minimum latency to a single gamestate, and allows a centeral place for lag compensation. bzflag does little lag compensation, and could still be more eficent. UDP is best for time sensitive data, but you have to do more then just send via UDP.

[Dameon1]

I have been considering various P2P mechanicsms for realtime situations such as this.

The biggest concern is indeed latency. If a P2P mechanism is to be used, while still maintaining the lowest possible latency, you must increase the number of connections per user while decreasing the number of hops. Of course, using such a mechanism while a central server is still the authority you will always have higher latency. But what about sacrificing latency for bandwidth?

My proposed system would be to have several Superclients (note: not peers) per server, as many as possible really. These superclients are capable of accepting incoming TCP connections/incoming UDP, have high bandwidth, annd preferably have low latency to/from the server. When a regular client connects, it will establish and maintain a connection to the server. The server will reccomend between zero and two superclients to connect to, which the client will optionally do. Note that a connection to a superclient is one way, save initial handshaking.

Topography is great, but why implement this in the first place? Remember that your average internet connection can download tons (as if electrons weighed much) more than it can upload. In BZFLAG, a server is constantly sending data to a client. This data is not particularly large in size, but multiply that by the number of players and all of a sudden your server is bottlenecked by its pipe. Another thing to note about BZFLAG is that a hefty chunk of what the server sends to each client is not only optional (eg movement) but the same for each client as well! Sure, it's great to know that Tank X is at Location Y, but if your client didn't receive that, he would still "be there" via dead reckoning. My suggestion to solve this through my topographical example is simple enough. Give the Superclients priority on the server side. When the server needs to send an update (In particular, the type mentioned above) it will start sending to its superclients, then to its regular clients, stopping when it has reached some sort of limit. This limit should be somehow set by a Quality of Service algoriithm to make sure the server doesn't waste all its pipe in one spurt. In this situation, when the server runs out of bandwidth, Client Z may not have gotten anything from the server, but will shortly from the Superclient it is connected to. Client X gets it from Superclient and Server, and in the event the Superclient attempted to modify the data, the server can be notified (A majority vote by the Superclient's clients will get it banned/demoted).

The Final Effect:
By using a system such as this, a BZFLAG server can support more clients on a given connection. In some cases, this could allow more frequent updates to each client since the server is conserving bandwidth. With its innate cheating prevention, you get the best of P2P and Client-Server: The server is always authoritave, but to more clients.

Is the concept solid? If so, I have some more ways to expand on this.

[toaster]

No, it's not really solid, because you get two levels of lag in the game. You get lag between the "superclient" and the server, and then lag between the player and the "superclient."

There are plenty of references to indicate the amount of bandwidth needed for a server to run and support 20 - 25 users. About 256kbps does the trick. The average home broadband is almost good enough, except that it's too broadly shared by your network neighbors and therefor not quite sufficiently consistent for large numbers of players. A lot of the servers are run at ISPs, or on college networks. These have bandwidth available, and perform fine. The problem with these is that other communications lines are congested.

For example, I can see a slowdown in my broadband provider every weekday afternoon around 4:00. That's when all the kids get home from school, and when parents start getting home from work, and they jump on the Internet to do their thing. My lag, over a period of 15 minutes, might move from an average of 75ms to a given server, all the way up to 400ms or higher.

The server is usually still just fine, just humming along. Putting a "superclient" between my client and a server would just add another lag level to it. If I have high bandwidth access to the superclient and it has high bandwidth access to the server, then I easily also have high bandwidth access to the server. If my bandwidth is low, why add a superclient in between?

You need game state management to be as close to real time as possible, to keep out the effects of disappearing or warping tanks, jitters, and the various other problems you see arise on live servers when players are lagging.

Also, implementing server QoS adds overhead on the server. There is really no need. In reality, what this concept would do would be to provide players with good access better service, to the detriment of other clients.

[RPG]

I think the way it's run now, client/server UDP configuration, is just fine.

[JeffM]

bandwith isn't the problem, it's latency.

smarter packet queing on the server side with some throttleing can help make the game have more players ( prevent flooding ). But you still need the lowest latency connection you can get. Like the Toaster says, any thing that adds latency is BAD.

[1veedo]

Dameon1 has the same concept I originally talked about.