Pantropy – Massive Performance Boost by Using NetworkArrays / Photon Bolt
This guest post was written by Julian Kaulitzki, lead developer for Pantropy at indie game studio Brain Stone.
Pantropy is a faction-based, SciFi-themed mech-game for PC and is, at the time of writing, in closed-alpha. Pantropy will eventually allow up to 128 players in one joint session. Conquer an alien world, farm ores, build bases and craft various items such as weapons and vehicles to defend yourself from players of the other faction and AI controlled mobs.
For more information about the game please check out Pantropy’s Kickstarter campaign.
Game Performance Issues During Pre-Alpha
In late 2016, we publicly launched a pre-alpha demo where players would test various features of our game. One of these features was building your own base using parts like foundations, walls, floors, generators, furnaces, and many others. During the pre-alpha we noticed that the server and client performance declined over time. Sometimes it got so bad that we had to wipe the servers.
We found that the frame rate was proportional to the amount of active BoltEntities on a server after some debugging. So with 10 players online, where each one had a base that likely exceeded 100 parts and each part having a BoltEntity attached, we ended up with 1000+ active BoltEntities at a time. Quite a lot.
(Quick disclaimer: we did not utilize scoping/freezing/idling/priority calculation during that time.)
The Solution
We were in need of a system that would use less BoltEntities without compromising the gameplay or the size of bases. The parts you would use to construct your base can be split into two main groups:
Passive parts – These parts only need to synchronize the following values over the network: Position, Rotation, Structural Integrity, Health and a PrefabID. For example: foundations, pillars, walls and, floors.
Active parts – These parts need to synchronize more values than what is listed above. Parts like Generators, Furnaces, Storage Boxes and, Crafting-Stations store items and some extra values regarding electricity.
95% of a base consists of passive parts and, since they need to synchronize very little information, we figured tostore their data in a NetworkArray_Objects instead of individual BoltEntities.
A NetworkArray_Objects is a bolt class that stores BoltObjects in an array on a state.
In this case the NetworkArray stores BoltObjects of type BaseElement_Obj.
Each of these BoltObjects represents a passive part of a base, so, naturally, a BaseElement_Obj stores the following values:
Position (Vector3) – Where the BasePart is located in World Space.
Rotation (Quaternion) – The orientation of the BasePart in World Space.
Structural Integrity (float) – How stable is this part.
Health (int) – How much health the part has.
PrefabID (int) – Not to be confused with the struct that comes with bolt that is also called PrefabID. This value is used to identify part types.
IsSpawned (bool) – Determines if this entry of the NetworkArray is a unused.
Bolt limits each state to a maximum of 1024 properties so the array can only be 170 entries long.
The BaseElementsManager is a script that inherits from Bolt’s EntityBehavior. It has on its state a property called “BaseElements” on its state that isa NetworkArray of type BaseElement_Obj.(see picture above).
Now here comes the important part:
Its main function is to store the data of passive parts and to handle the proxys of this data.
A proxy is a simple GameObject, without a BoltEntity attached, that represents an entry in the NetworkArray.
So every entry that is marked as “IsSpawned = True” has a GameObject(the proxy) in the world with which the players can interact. In this case the proxies are parts of a base.
A base can be made from more than one of these BaseElementsManagers so we need another script that manages all of these. The BaseManager script handles requests to add/remove/alter a passive part and manages the BaseElementsManagers of one base. For example, if we want to add a new part to the base but the base already has 170 passive parts-which means that the current BaseElementsManager‘s NetworkArray is full-a new BaseElementsManager is spawned and used to store the data of the new part.
The callback OnBaseElementsStateArrayChanged inside of the BaseElementsManager script is then called hence the NetworkArray “BaseElements“ changed. The HandleBaseElement method is then called by passing the arrayIndex of the changed entry inside of the callback. The method HandleBaseElement then checks if the changed entry is marked as “IsSpawned = True” and, if this is the case, it checks if there is already a proxy; for that entry. If there is no proxy for that entry then it spawns one.
However, if the entry is marked as “IsSpawned = False” and the entry has a proxy, then this proxy will be removed.
private void OnBaseElementsStateArrayChanged(IState s, string path, ArrayIndices indices)
{
int ArrayIndex = indices[0];
HandleBaseElement(ArrayIndex);
}
private void HandleBaseElement(int arrayIndex)
{
// state.BaseElements is the NetworkArray of type BaseElement_Obj
BaseElement_Obj element_Obj = state.BaseElements[arrayIndex];
if(element_Obj.IsSpawned)
{
// ProxyBaseElements is an array of type BaseElement with a length equivalent to the NetworkArray
if (ProxyBaseElements[arrayIndex] == null)
SpawnBaseElementProxy(element_Obj, arrayIndex);
}else
{
if (ProxyBaseElements[arrayIndex] != null)
RemoveBaseElementProxy(ProxyBaseElements[arrayIndex], arrayIndex);
}
}Keep in mind that the callback to the NetworkArray is called whenever an entry’s value changes. So when the Health of a BaseElement_Obj inside of the NetworkArray is changed the callback is called.
Some Tweaks
In order to prevent memory leaks, we pool our proxys. We have a routine running on each BaseElementsManager that checks if the player is nearby and if so it should spawn a proxy. If not, then it should add the already spawned proxy back into the pool so another base can use it.
Whenever a new part is added/removed we calculate the average position of all of the passive parts to get the center point of the base. We then move the BaseManager and BaseElementsManagers to this point. Now we can use their positions for scoping/freezing/idling/priority calculations.
Conclusion
Using proxies to represent data stored in a NetworkArray in combination with scoping/freezing/idling/priority calculations is more efficient and easier than managing thousands of individual BoltEntities.
With this system, we were able to reduce the BoltEntity count for passive parts of ten bases from 1700 to only 10!
The same system finds place in our ores. We have 5000 ores scattered around the world and every one of them was originally an individual BoltEntity. We are now using 15 OreNodesManagers each one has a BoltEntity attached and manages 340 ore nodes.
You could also use this system to scatter trees around the world using a seed and manage 500 trees with only one BoltEntity.
Here are some numbers on how the performance improved with the proxy system:
I created a little script that spawns a cube that is made out of 3380 individual parts.
The cube’s dimensions: 13w x 13l x 20h
In the first run of the test, every part had a BoltEntity attatched. Each entity was active meaning it was not in idle or frozen. This resulted in an overhead of 32.20 ms caused by Bolt’s BoltPoll.FixedUpdate loop.(highlighted in the screenshot below)
The second testrun was performed using my Proxy system. So there were 25 BaseElementsManagers each one managed 140 proxys. This means that the entire cube consits of only 25 BoltEntities. Each entity was active meaning it was not in idle or frozen. The result of that was 0.69 ms of Bolt’s BoltPoll.FixedUpdate loop. (highlighted in the screenshot below)
