Respond to Connection state might be too large log messages

Problem description: You see “error when serializing for state sync” messages with a “connection state might be too large” clarification log entries for a Engine Cluster. You might also experience intermittent or continuous problems with clustering and traffic flow, which are typically alleviated for some time by rebooting all clustered nodes.

Reason: The Engine keeps a record of all connections that are handled statefully to be able to track the connection. When the Engine is clustered, this connection table must be synchronized between the nodes to allow connections to continue if a node goes down. When the state table grows excessively large, the Engines can no longer effectively use it.

A misconfiguration usually causes this message. Typical configuration problems include:

• Using the Oracle Protocol Agent on the actual database connections between the client and the server. The Oracle Protocol Agent is meant for cases where TCP port 1521 is used only for negotiating the port number for Oracle database connections. The port number for the actual connection is assigned dynamically. The Oracle Protocol Agent must not be used in any other cases.
• Excessive idle timeouts defined in Access Rules. All TCP connections are normally explicitly closed by the communicating parties and can therefore be cleared from the state table based on actual connection state. Non-TCP protocols do not establish connections. The communications are still handled as virtual connections on the Engine to allow all Engine features to be used on the traffic. Because the communicating parties do not have a closing mechanism, these virtual connections are never cleared from the Engines’ connection records before the communications are left idle (unused) for the duration of the defined timeout. If Access rules define excessively long timeouts for such traffic between many different hosts, the connection state table can grow very large.