inConcert troubleshooting

Heading

Detailed Guide to Stop inConcert

1. Safely stop all components of inConcert:

/usr/share/inconcert/services/stop.sh

1. Stop additional components used by inConcert:

sv stop /etc/service/*

1. Stop the Cassandra Database Engine:

/etc/init.d/cassandra  stop

1. Validate that all inConcert services are stopped

sv  status    /etc/service/*

You  can validate Cassandra's  state with:

/etc/init.d/cassandra  status

 

Step-by-step guide to starting inConcert

1. Start the Cassandra Database Engine:

   /etc/init.d/cassandra star

Its a good idea to wait a couple of minutes before proceeding, so that the database engine starts completely before the inConcert services start.

It can be executed:

nodetool  status

And from the log confirm that it is ready to receive connections

tail   -F    /var/log/cassandra/system.log and  search for the message  "Listening for thrift clients"

Until you get:

xss     =       -ea     -javaagent:/usr/share/cassandra/lib/jamm-0.2.5.jar                                   -XX:+UseThreadPriorities                                                                                                                           -

XX:ThreadPriorityPolicy=42             -Xms1024M      -Xmx1024M     -Xmn100M     -XX:+HeapDumpOnOut0fMemoryError                       -

Xss180k

Datacenter:  datacenterl

 

Status=Up/Down

1/  State=Normal/Leaving/Joining/Moving

--        Address               Load                    Tokens   Owns      (effective)               Host  ID

Rack

UN    10.2.14.4            102.85 MB      256               100.0%                                    ccaf188d-bb9b-41c4-ac6c-3af479df3b29

rack1

 

1. Start additional components used by inConcert:

sv  start  redis

* Clean the  RealTimeKeys.

redis-cli           -p    6377  KEYS  "RealTime::*" | sed  's/.*/"&"/'   |    1 xargs  redis-cli            -p    6377 DEL

when there  is  authentication  in  redis:   'xxx'  is the  authentication  password

redis-cli           -p    6377    -a  'xxx'         --no-auth-warning  KEYS  "RealTime::*"   |   sed  's/.*/"&"/'  |   xargs

redis-cli           -p    6377    -a  'xxx'         --no-auth-warning DEL

 

*  Clean  configuration_server

redis-cli           -p    6377  KEYS  "CONFIGURATION_SERVER::SESSIONS_TIMESTAMP"  |  sed  's/.*/"&"/'   |  xargs

redis-cli           -p    6377 DEL

when there  is  authentication  in  redis:   'xxx'  is the  authentication  password

redis-cli           -p    6377    -a  'xxx'         --no-auth-warning  KEYS  "CONFIGURATION_SERVER::SESSIONS_TIMESTAMP"

|sed  's/.*/"&"/'           |         xargs  redis-cli             -p    6377    -a  'xxx'         --no-auth-warning DEL

sv  start  redis-agentservice

sv  start  redis-outboundengine

sv  start webdesigner

sv  start  configserver

sv  start  elasticsearch

sv  start  allegroproxy

*  redis-outboundengine only if it  is  build       2610 or higher

*  elasticsearch only on the first  node

/usr/share/inconcert/services/start.sh

1. Start inConcert Services:

Finally,  upload the  nginx  service  since everything else  is  up sv start  nginx

 

In case of restarting only the Asterisk service, it must be taken into account that if this is done in

production (with connected agents) it is very likely that Fail2Ban will generate a ban on the IP of the agents.

To avoid this situation, restart the service following these steps:

 

Shut down the Fail2Ban service:

service fail2ban  stop

Shut down the Asterisk service:

sv  stop asterisk

Start up the Asterisk service:

sv  start  asterisk

Wait 10-15 seconds.

Start up the Fail2Ban service:

service fail2ban  start