Load Balancing

When the processing limit of a Speech Recognition Server instance is reached, and it is not enough to meet the demand of the client applications, the system’s capacity must be increased by adding new instances to the environment. This is accomplished by using a software to distribute and balance the load of the client applications among the recognition server’s instances.

This guide presents a very simple topology to create a Speech Recognition cluster, using Apache HTTP Server as a load balancer. The figure below illustrates this topology. This is merely an example and can be altered, using more complex balancing schemes to provide high availability, eliminating failure (single point of failure).

../_images/balanceador.png

Fig. 6 Speech recognition cluster topology.

Installation requirements.

To install an Apache HTTP server in a dedicated machine for load balancing requires the following minimum configuration:

  • Hardware:

    • 1 GB free RAM

    • 100 MB free HD

    • 2 vCPUs @ 2.20 GHz

    • 100 Mbps network Interface

  • Software:

    • Recommended: Linux Red Hat 7 or CentOS 7

Apache installation

Our example was based on the Apache HTTP Server 2.4.6. To install the Apache, run:

$ sudo yum -y groupinstall "Web Server"

Configuration

Configure the load balancing cluster, creating the file /etc/httpd/conf.d/asr-balancer.conf:

<VirtualHost *:80>

ProxyRequests off

<Proxy balancer://asrcluster_ws>
    BalancerMember ws://<IP do ASR 1>:8025
    BalancerMember ws://<IP do ASR 2>:8025
    ProxySet lbmethod=bybusyness
</Proxy>

ProxyPass /asr-server/asr balancer://asrcluster_ws/asr-server/asr

<Proxy balancer://asrcluster_rest>
    BalancerMember http://<IP do ASR 1>:8025
    BalancerMember http://<IP do ASR 2>:8025
    ProxySet lbmethod=bybusyness
</Proxy>

ProxyPass /asr-server/rest/recognize balancer://asrcluster_rest/asr-server/rest/recognize

<Location "/asr-balancer">
    SetHandler balancer-manager
</Location>

</VirtualHost>

Execution

To launch the Apache HTTP Server:

$ sudo systemctl start httpd

To make sure the Apache HTTP Server is running correctly:

$ sudo systemctl status httpd

$ systemctl status httpd
● httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled)
   Active: active (running) since Fri 2018-08-10 21:01:50 UTC; 3s ago
     Docs: man:httpd(8)
           man:apachectl(8)
 Main PID: 170 (httpd)
   Status: "Processing requests..."
   CGroup: /docker/414f528c91a01e6217ada3e0e7c10dbcfffa9c4a2205043de3ef9f539d43cc0d/system.slice/httpd.service
           ├─170 /usr/sbin/httpd -DFOREGROUND
           ├─171 /usr/sbin/httpd -DFOREGROUND
           ├─172 /usr/sbin/httpd -DFOREGROUND
           ├─173 /usr/sbin/httpd -DFOREGROUND
           ├─174 /usr/sbin/httpd -DFOREGROUND
           ├─175 /usr/sbin/httpd -DFOREGROUND
           └─176 /usr/sbin/httpd -DFOREGROUND

Aug 10 21:01:50 414f528c91a0 systemd[1]: Starting The Apache HTTP Server...
Aug 10 21:01:50 414f528c91a0 systemd[1]: Started The Apache HTTP Server.

To stop the Apache HTTP Server:

$ sudo systemctl stop httpd

Access

The load balancing server must be addressed using the same URLs of the WebSocket and REST APIs of the CPQD ASR. Just change the IP for the balancer and the access port to port 80 (in this case, you don’t even need to write it).

ASR WebSocket API:

ws:<IP_Balanceador>/asr-server/asr

ASR REST API:

http:<IP_Balanceador>/asr-server/rest/recognize

The Balancing Monitor Web page can be accessed at:

http://<IP_Balanceador>/asr-balancer