Page 1 of 1

screens are missing only 5 screen are showing

PostPosted: Tue Mar 21, 2023 4:11 am
by shameemvallanchira
Hi All,
i am new to vicidial and using an old version.my issue started suddenly all agent went logout and they are unable to login in again. the agent login call to the soft phone is not happening. i have compared the screens with another working server and found that the only 5 screens are active out of the 11 screens. i tried the following command "perl /usr/share/astguiclient/ADMIN_keepalive_ALL.pl --debugX " and it seems some of the screens are not getting up.my can any please help me on this?

VERSION: 2.2.1-237
BUILD: 100510-2015
ASTERISK : 1.4.39.1

starting AST_update...
starting AST_manager_send...
starting AST_manager_listen...
starting AST_VDauto_dial...
starting AST_VDremote_agents...


manager.conf

[general]
enabled = yes
port = 5038
bindaddr = 0.0.0.0

[cron]
secret = 1234
read = system,call,log,verbose,command,agent,user
write = system,call,log,verbose,command,agent,user

[updatecron]
secret = 1234
read = command
write = command

[listencron]
secret = 1234
read = system,call,log,verbose,command,agent,user
write = command

[sendcron]
secret = 1234
read = command
write = system,call,log,verbose,command,agent,user




[root@capstone ~]# perl /usr/share/astguiclient/ADMIN_keepalive_ALL.pl --debugX

----- SUPER-DUPER DEBUGGING -----

AST_update set to keepalive
AST_send_listen set to keepalive
AST_VDauto_dial set to keepalive
AST_VDremote_agents set to keepalive
AST_VDadapt set to keepalive
FastAGI_log set to keepalive
ip_relay set to keepalive
0| 1 /sbin/init|
1| 2 [kthreadd]|
2| 3 [migration/0]|
3| 4 [ksoftirqd/0]|
4| 5 [stopper/0]|
5| 6 [watchdog/0]|
6| 7 [migration/1]|
7| 8 [stopper/1]|
8| 9 [ksoftirqd/1]|
9| 10 [watchdog/1]|
10| 11 [migration/2]|
11| 12 [stopper/2]|
12| 13 [ksoftirqd/2]|
13| 14 [watchdog/2]|
14| 15 [migration/3]|
15| 16 [stopper/3]|
16| 17 [ksoftirqd/3]|
17| 18 [watchdog/3]|
18| 19 [events/0]|
19| 20 [events/1]|
20| 21 [events/2]|
21| 22 [events/3]|
22| 23 [events/0]|
23| 24 [events/1]|
24| 25 [events/2]|
25| 26 [events/3]|
26| 27 [events_long/0]|
27| 28 [events_long/1]|
28| 29 [events_long/2]|
29| 30 [events_long/3]|
30| 31 [events_power_ef]|
31| 32 [events_power_ef]|
32| 33 [events_power_ef]|
33| 34 [events_power_ef]|
34| 35 [cgroup]|
35| 36 [khelper]|
36| 37 [netns]|
37| 38 [async/mgr]|
38| 39 [pm]|
39| 40 [sync_supers]|
40| 41 [bdi-default]|
41| 42 [kintegrityd/0]|
42| 43 [kintegrityd/1]|
43| 44 [kintegrityd/2]|
44| 45 [kintegrityd/3]|
45| 46 [kblockd/0]|
46| 47 [kblockd/1]|
47| 48 [kblockd/2]|
48| 49 [kblockd/3]|
49| 50 [kacpid]|
50| 51 [kacpi_notify]|
51| 52 [kacpi_hotplug]|
52| 53 [ata_aux]|
53| 54 [ata_sff/0]|
54| 55 [ata_sff/1]|
55| 56 [ata_sff/2]|
56| 57 [ata_sff/3]|
57| 58 [ksuspend_usbd]|
58| 59 [khubd]|
59| 60 [kseriod]|
60| 61 [md/0]|
61| 62 [md/1]|
62| 63 [md/2]|
63| 64 [md/3]|
64| 65 [md_misc/0]|
65| 66 [md_misc/1]|
66| 67 [md_misc/2]|
67| 68 [md_misc/3]|
68| 69 [linkwatch]|
69| 72 [khungtaskd]|
70| 73 [kswapd0]|
71| 74 [ksmd]|
72| 75 [khugepaged]|
73| 76 [aio/0]|
74| 77 [aio/1]|
75| 78 [aio/2]|
76| 79 [aio/3]|
77| 80 [crypto/0]|
78| 81 [crypto/1]|
79| 82 [crypto/2]|
80| 83 [crypto/3]|
81| 90 [kthrotld/0]|
82| 91 [kthrotld/1]|
83| 92 [kthrotld/2]|
84| 93 [kthrotld/3]|
85| 95 [kpsmoused]|
86| 96 [usbhid_resumer]|
87| 97 [deferwq]|
88| 129 [kdmremove]|
89| 130 [kstriped]|
90| 165 [i915]|
91| 166 [i915-dp]|
92| 167 [i915-hangcheck]|
93| 398 [scsi_eh_0]|
94| 399 [scsi_eh_1]|
95| 400 [scsi_eh_2]|
96| 401 [scsi_eh_3]|
97| 402 [scsi_eh_4]|
98| 403 [scsi_eh_5]|
99| 663 [jbd2/sda2-8]|
100| 664 [ext4-dio-unwrit]|
101| 760 /sbin/udevd -d|
102| 931 [flush-8:0]|
103| 1223 [jbd2/sda1-8]|
104| 1224 [ext4-dio-unwrit]|
105| 1322 [kauditd]|
106| 1380 [ib_addr]|
107| 1385 [infiniband/0]|
108| 1386 [infiniband/1]|
109| 1387 [infiniband/2]|
110| 1388 [infiniband/3]|
111| 1397 [ib_mcast]|
112| 1402 [iw_cm_wq]|
113| 1407 [ib_cm/0]|
114| 1409 [ib_cm/1]|
115| 1410 [ib_cm/2]|
116| 1411 [ib_cm/3]|
117| 1416 [rdma_cm]|
118| 1437 [ipoib_flush]|
119| 1661 auditd|
120| 1695 /sbin/rsyslogd -i /var/run/syslogd.pid -c 5|
121| 1720 [kondemand/0]|
122| 1721 [kondemand/1]|
123| 1722 [kondemand/2]|
124| 1723 [kondemand/3]|
125| 1795 irqbalance --pid=/var/run/irqbalance.pid|
126| 1813 rpcbind|
127| 1835 rpc.statd|
128| 1869 dbus-daemon --system|
129| 1891 cupsd -C /etc/cups/cupsd.conf|
130| 1923 /usr/sbin/acpid|
131| 1935 hald|
132| 1936 hald-runner|
133| 1968 hald-addon-input: Listening on /dev/input/event0 /dev/input/event1 /dev/input/event4 /dev/input/event2|
134| 1976 /usr/libexec/hald-addon-generic-backlight|
135| 1984 hald-addon-acpi: listening on acpid socket /var/run/acpid.socket|
136| 2008 automount --pid-file /var/run/autofs.pid|
137| 2128 /usr/sbin/mcelog --daemon|
138| 2145 /usr/sbin/sshd|
139| 2156 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g|
140| 2235 /usr/libexec/postfix/master|
141| 2242 qmgr -l -t fifo -u|
142| 2249 /usr/sbin/abrtd|
143| 2305 crond|
144| 2321 /usr/sbin/atd|
145| 2337 /usr/sbin/certmonger -S -p /var/run/certmonger.pid|
146| 2414 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --pid-file=/var/run/mysqld/mysqld.pid --basedir=/usr --user=mysql|
147| 2542 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock|
148| 2683 /usr/bin/SCREEN -d -m -S ASTfastlog /usr/share/astguiclient/FastAGI_log.pl --debug|
149| 2692 /usr/bin/perl /usr/share/astguiclient/FastAGI_log.pl --debug|
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug|
150| 2704 /usr/sbin/httpd|
151| 2715 /usr/bin/perl /usr/share/astguiclient/FastAGI_log.pl --debug|
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug|
152| 2728 /usr/sbin/httpd|
153| 2729 /usr/sbin/httpd|
154| 2730 /usr/sbin/httpd|
155| 2731 /usr/sbin/httpd|
156| 2732 /usr/sbin/httpd|
157| 2733 /usr/sbin/httpd|
158| 2734 /usr/sbin/httpd|
159| 2735 /usr/sbin/httpd|
160| 2742 /usr/bin/SCREEN -S astshell20230319133506|
161| 2743 /bin/sh|
162| 2757 SCREEN -L -S asterisk|
163| 2758 /bin/sh|
164| 2793 /sbin/mingetty /dev/tty1|
165| 2795 /sbin/mingetty /dev/tty2|
166| 2797 /sbin/mingetty /dev/tty3|
167| 2799 /sbin/mingetty /dev/tty4|
168| 2801 /sbin/mingetty /dev/tty5|
169| 2803 /sbin/mingetty /dev/tty6|
170| 2804 /sbin/udevd -d|
171| 2805 /sbin/udevd -d|
172| 3367 cleanup -z -t unix -u|
173| 4447 pickup -l -t fifo -u|
174| 4765 local -t unix|
175| 5595 /usr/bin/perl /usr/share/astguiclient/FastAGI_log.pl --debug|
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug|
176| 6272 trivial-rewrite -n rewrite -t unix -u|
177| 6274 cleanup -z -t unix -u|
178| 6950 sshd: root@pts/0 |
179| 6952 -bash|
180| 6997 bounce -z -t unix -u|
181| 7177 local -t unix|
182| 7243 bounce -z -t unix -u|
183| 7684 CROND|
184| 7689 /usr/bin/perl /usr/share/astguiclient/AST_manager_kill_hung_congested.pl|
185| 7855 perl /usr/share/astguiclient/ADMIN_keepalive_ALL.pl --debugX|
186| 7856 /bin/ps -o %p %a --no-headers -A|
187| 9271 /usr/bin/perl /usr/share/astguiclient/FastAGI_log.pl --debug|
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug|
188|16350 /usr/bin/perl /usr/share/astguiclient/FastAGI_log.pl --debug|
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug|
189|22397 /bin/sh /usr/sbin/safe_asterisk|
190|22400 /usr/sbin/asterisk -f -vvvg -c|
191|24823 /usr/bin/SCREEN -d -m -S ASTVDadapt /usr/share/astguiclient/AST_VDadapt.pl --debug|
192|24825 /usr/bin/perl /usr/share/astguiclient/AST_VDadapt.pl --debug|
AST_VDadapt RUNNING: |/usr/share/astguiclient/AST_VDadapt.pl --debug|
193|26171 /usr/sbin/httpd|
194|26932 /usr/bin/perl /usr/share/astguiclient/FastAGI_log.pl --debug|
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug|
double check that processes are not running...
0| 1 /sbin/init|
1| 2 [kthreadd]|
2| 3 [migration/0]|
3| 4 [ksoftirqd/0]|
4| 5 [stopper/0]|
5| 6 [watchdog/0]|
6| 7 [migration/1]|
7| 8 [stopper/1]|
8| 9 [ksoftirqd/1]|
9| 10 [watchdog/1]|
10| 11 [migration/2]|
11| 12 [stopper/2]|
12| 13 [ksoftirqd/2]|
13| 14 [watchdog/2]|
14| 15 [migration/3]|
15| 16 [stopper/3]|
16| 17 [ksoftirqd/3]|
17| 18 [watchdog/3]|
18| 19 [events/0]|
19| 20 [events/1]|
20| 21 [events/2]|
21| 22 [events/3]|
22| 23 [events/0]|
23| 24 [events/1]|
24| 25 [events/2]|
25| 26 [events/3]|
26| 27 [events_long/0]|
27| 28 [events_long/1]|
28| 29 [events_long/2]|
29| 30 [events_long/3]|
30| 31 [events_power_ef]|
31| 32 [events_power_ef]|
32| 33 [events_power_ef]|
33| 34 [events_power_ef]|
34| 35 [cgroup]|
35| 36 [khelper]|
36| 37 [netns]|
37| 38 [async/mgr]|
38| 39 [pm]|
39| 40 [sync_supers]|
40| 41 [bdi-default]|
41| 42 [kintegrityd/0]|
42| 43 [kintegrityd/1]|
43| 44 [kintegrityd/2]|
44| 45 [kintegrityd/3]|
45| 46 [kblockd/0]|
46| 47 [kblockd/1]|
47| 48 [kblockd/2]|
48| 49 [kblockd/3]|
49| 50 [kacpid]|
50| 51 [kacpi_notify]|
51| 52 [kacpi_hotplug]|
52| 53 [ata_aux]|
53| 54 [ata_sff/0]|
54| 55 [ata_sff/1]|
55| 56 [ata_sff/2]|
56| 57 [ata_sff/3]|
57| 58 [ksuspend_usbd]|
58| 59 [khubd]|
59| 60 [kseriod]|
60| 61 [md/0]|
61| 62 [md/1]|
62| 63 [md/2]|
63| 64 [md/3]|
64| 65 [md_misc/0]|
65| 66 [md_misc/1]|
66| 67 [md_misc/2]|
67| 68 [md_misc/3]|
68| 69 [linkwatch]|
69| 72 [khungtaskd]|
70| 73 [kswapd0]|
71| 74 [ksmd]|
72| 75 [khugepaged]|
73| 76 [aio/0]|
74| 77 [aio/1]|
75| 78 [aio/2]|
76| 79 [aio/3]|
77| 80 [crypto/0]|
78| 81 [crypto/1]|
79| 82 [crypto/2]|
80| 83 [crypto/3]|
81| 90 [kthrotld/0]|
82| 91 [kthrotld/1]|
83| 92 [kthrotld/2]|
84| 93 [kthrotld/3]|
85| 95 [kpsmoused]|
86| 96 [usbhid_resumer]|
87| 97 [deferwq]|
88| 129 [kdmremove]|
89| 130 [kstriped]|
90| 165 [i915]|
91| 166 [i915-dp]|
92| 167 [i915-hangcheck]|
93| 398 [scsi_eh_0]|
94| 399 [scsi_eh_1]|
95| 400 [scsi_eh_2]|
96| 401 [scsi_eh_3]|
97| 402 [scsi_eh_4]|
98| 403 [scsi_eh_5]|
99| 663 [jbd2/sda2-8]|
100| 664 [ext4-dio-unwrit]|
101| 760 /sbin/udevd -d|
102| 931 [flush-8:0]|
103| 1223 [jbd2/sda1-8]|
104| 1224 [ext4-dio-unwrit]|
105| 1322 [kauditd]|
106| 1380 [ib_addr]|
107| 1385 [infiniband/0]|
108| 1386 [infiniband/1]|
109| 1387 [infiniband/2]|
110| 1388 [infiniband/3]|
111| 1397 [ib_mcast]|
112| 1402 [iw_cm_wq]|
113| 1407 [ib_cm/0]|
114| 1409 [ib_cm/1]|
115| 1410 [ib_cm/2]|
116| 1411 [ib_cm/3]|
117| 1416 [rdma_cm]|
118| 1437 [ipoib_flush]|
119| 1661 auditd|
120| 1695 /sbin/rsyslogd -i /var/run/syslogd.pid -c 5|
121| 1720 [kondemand/0]|
122| 1721 [kondemand/1]|
123| 1722 [kondemand/2]|
124| 1723 [kondemand/3]|
125| 1795 irqbalance --pid=/var/run/irqbalance.pid|
126| 1813 rpcbind|
127| 1835 rpc.statd|
128| 1869 dbus-daemon --system|
129| 1891 cupsd -C /etc/cups/cupsd.conf|
130| 1923 /usr/sbin/acpid|
131| 1935 hald|
132| 1936 hald-runner|
133| 1968 hald-addon-input: Listening on /dev/input/event0 /dev/input/event1 /dev/input/event4 /dev/input/event2|
134| 1976 /usr/libexec/hald-addon-generic-backlight|
135| 1984 hald-addon-acpi: listening on acpid socket /var/run/acpid.socket|
136| 2008 automount --pid-file /var/run/autofs.pid|
137| 2128 /usr/sbin/mcelog --daemon|
138| 2145 /usr/sbin/sshd|
139| 2156 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g|
140| 2235 /usr/libexec/postfix/master|
141| 2242 qmgr -l -t fifo -u|
142| 2249 /usr/sbin/abrtd|
143| 2305 crond|
144| 2321 /usr/sbin/atd|
145| 2337 /usr/sbin/certmonger -S -p /var/run/certmonger.pid|
146| 2414 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --pid-file=/var/run/mysqld/mysqld.pid --basedir=/usr --user=mysql|
147| 2542 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock|
148| 2683 /usr/bin/SCREEN -d -m -S ASTfastlog /usr/share/astguiclient/FastAGI_log.pl --debug|
149| 2692 /usr/bin/perl /usr/share/astguiclient/FastAGI_log.pl --debug|
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug|
150| 2704 /usr/sbin/httpd|
151| 2715 /usr/bin/perl /usr/share/astguiclient/FastAGI_log.pl --debug|
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug|
152| 2728 /usr/sbin/httpd|
153| 2729 /usr/sbin/httpd|
154| 2730 /usr/sbin/httpd|
155| 2731 /usr/sbin/httpd|
156| 2732 /usr/sbin/httpd|
157| 2733 /usr/sbin/httpd|
158| 2734 /usr/sbin/httpd|
159| 2735 /usr/sbin/httpd|
160| 2742 /usr/bin/SCREEN -S astshell20230319133506|
161| 2743 /bin/sh|
162| 2757 SCREEN -L -S asterisk|
163| 2758 /bin/sh|
164| 2793 /sbin/mingetty /dev/tty1|
165| 2795 /sbin/mingetty /dev/tty2|
166| 2797 /sbin/mingetty /dev/tty3|
167| 2799 /sbin/mingetty /dev/tty4|
168| 2801 /sbin/mingetty /dev/tty5|
169| 2803 /sbin/mingetty /dev/tty6|
170| 2804 /sbin/udevd -d|
171| 2805 /sbin/udevd -d|
172| 3367 cleanup -z -t unix -u|
173| 4447 pickup -l -t fifo -u|
174| 4765 local -t unix|
175| 5595 /usr/bin/perl /usr/share/astguiclient/FastAGI_log.pl --debug|
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug|
176| 6272 trivial-rewrite -n rewrite -t unix -u|
177| 6274 cleanup -z -t unix -u|
178| 6950 sshd: root@pts/0 |
179| 6952 -bash|
180| 6997 bounce -z -t unix -u|
181| 7177 local -t unix|
182| 7243 bounce -z -t unix -u|
183| 7684 CROND|
184| 7689 /usr/bin/perl /usr/share/astguiclient/AST_manager_kill_hung_congested.pl|
185| 7855 perl /usr/share/astguiclient/ADMIN_keepalive_ALL.pl --debugX|
186| 7858 /bin/ps -o %p %a --no-headers -A|
187| 9271 /usr/bin/perl /usr/share/astguiclient/FastAGI_log.pl --debug|
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug|
188|16350 /usr/bin/perl /usr/share/astguiclient/FastAGI_log.pl --debug|
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug|
189|22397 /bin/sh /usr/sbin/safe_asterisk|
190|22400 /usr/sbin/asterisk -f -vvvg -c|
191|24823 /usr/bin/SCREEN -d -m -S ASTVDadapt /usr/share/astguiclient/AST_VDadapt.pl --debug|
192|24825 /usr/bin/perl /usr/share/astguiclient/AST_VDadapt.pl --debug|
AST_VDadapt RUNNING: |/usr/share/astguiclient/AST_VDadapt.pl --debug|
193|26171 /usr/sbin/httpd|
194|26932 /usr/bin/perl /usr/share/astguiclient/FastAGI_log.pl --debug|
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug|
starting AST_update...
starting AST_manager_send...
starting AST_manager_listen...
starting AST_VDauto_dial...
starting AST_VDremote_agents...
starting ip_relay through relay_control...
|SELECT count(*) from system_settings where timeclock_end_of_day LIKE "%1252%";|
RESET LIST: 0|1252DONE

Re: screens are missing only 5 screen are showing

PostPosted: Tue Mar 21, 2023 1:30 pm
by jamiemurray
It looks like asterisk isn't running, some of the other screens REQUIRE asterisk to be running to remain up.
You could try the following to completely reload it without a full reboot:

Code: Select all
pkill screen && /usr/share/astguiclient/start_asterisk_boot.pl && /usr/share/astguiclient/ADMIN_keepalive_ALL.pl && /usr/share/astguiclient/ADMIN_keepalive_ALL.pl


Then check the output of
Code: Select all
screen -ls

again to see if all your usual screens are running, if so then you should be good to go.

p.s. enabling Auto-Restart asterisk on the servers page will cause the keepalive script to automatically restart asterisk if it crashes.

If you're facing repeated asterisk crashes though, you'll need to do further checks into the causes.