vicidial.org

by **okli** » Sat Dec 05, 2009 2:09 am

Played a bit with MySQL settings following several web pages and reduced wait_timeout to 30 secs.

AST_manager_kill_hung_congested.pl starts complaining as it opens connection, sleeps for 44 seconds(14+15+15) in total + processing time for the queries and script and then closes it.

Increasing wait_timeout surely fixes the issue, but wondered if it is a good idea to rely on and use an open connection for that long, especially in scenarios with MySQL on a separate box.

Are there other scripts in vicidial which rely on an open connection for long time, or should I simply put wait_timeout back to its default value and not play where should not?

Actually what is interesting for me is not that much the wait_timeout value, but whether from programming point of view it's right to use a connection to MySQL for that long, with sleeps for 15 secs. between queries. Newbie at that so please forgive me if the question is silly, just being curios.

by **mflorell** » Sat Dec 05, 2009 9:15 am

What I should really do is add logging to that script to see if it is even necessary with the newer versions of Asterisk 1.4. The whole reason it was written was to work around bugs in Asterisk 1.0 and 1.2 on higher load systems where calls would get stuck and not hangup after they were Congested.

As for setting your MySQL wait time so low, what was your reasoning for that?

by **okli** » Sat Dec 05, 2009 1:25 pm

mflorell wrote:...As for setting your MySQL wait time so low, what was your reasoning for that?

It started with random errors from Apache that cannot connect to the MySQL server, a couple of times a day, with loadavg like 0.20-0.40. Wait_timeout wasn't specifically needed, but just came across this option and thought it might be a good idea to get rid of idle connections more often. Then I noticed the issue with that script an became curious.
30 secs. was just a blind shot, following blindly several web pages:
http://www.notesbit.com/index.php/web-m ... -cnf-file/
http://www.interworx.com/forums/showthread.php?p=2346
http://www.mysqlperformanceblog.com/200 ... tallation/

I've increased it to 10000 afterward.

by **mflorell** » Sat Dec 05, 2009 4:43 pm

We usually just raise the max_connections up to 500 and that fixes it.

by **okli** » Sat Dec 05, 2009 5:18 pm

Initially I didn't want to raise that with this in mind:

Used MySQL memory = key_buffer + max_connections * (join_buffer + record_buffer + sort_buffer + thread_stack + tmp_table_size)

Thought it would be better idea to get rid of idle connections, rather than allowing more connections overall.
Eventually ended up the same way- with more allowed connections and long wait_timeout.
Server isn't loaded at all and has no more issues with random errors from Apache, so I wouldn't touch it anymore.

Thanks for your input, quite valuable as always.

by **mflorell** » Sat Dec 05, 2009 8:48 pm

I would say that formula isn't anywhere near accurate in reality.

As for Apache errors, we almost never have issues with Apache, do you have prefork configured with enough child processes?

by **okli** » Sat Dec 05, 2009 9:53 pm

Is this what you mean?

apache2.conf:

Code: Select all: ... <IfModule mpm_prefork_module> StartServers 5 MinSpareServers 5 MaxSpareServers 10 MaxClients 200 MaxRequestsPerChild 0 </IfModule> ... <IfModule mpm_worker_module> StartServers 2 MaxClients 200 MinSpareThreads 25 MaxSpareThreads 75 ThreadsPerChild 25 MaxRequestsPerChild 0 </IfModule> ...

by **mflorell** » Sun Dec 06, 2009 1:37 am

You might want to raise some of those a bit.

I wouldn't recommend these settings for an all-in-one server, but here's what we used for a standalone web server in a vicidial multi-server cluster:

<IfModule prefork.c>
ServerLimit 768
StartServers 450
MinSpareServers 250
MaxSpareServers 500
MaxClients 768
MaxRequestsPerChild 1000
</IfModule>

by **okli** » Sun Dec 06, 2009 3:29 am

For a combo Apache/MySQL server how much should I lower what you posted?
What would be balanced setting for memory allocation for Apache and MySQL? The box has 12GB RAM, 1 x QC Xeon Nehalem, 2 x 15k SAS in RAID1, serving 35-40 agents and looking for doubling them in a couple of months.

Where should I look for some statistic which may show a need to increase/decrease these values? There is nothing interesting in error.log.

by **mflorell** » Sun Dec 06, 2009 5:27 am

try cutting them in half and see how that works for you.

by **okli** » Sun Dec 06, 2009 2:25 pm

Thanks Matt, will do.

by **okli** » Tue Dec 08, 2009 10:19 pm

It seems Apache2 coming with Debian lenny is limited to 256 maxclients. Raising it above that apache throws:

WARNING: MaxClients of 300 exceeds ServerLimit value of 256 servers,
lowering MaxClients to 256. To increase, please see the ServerLimit
directive

Left it at 256, no issues for last few days.

by **mflorell** » Wed Dec 09, 2009 7:30 am

Sounds good.

We usually build Apache from source with extra modules so I can't say I've ever run into that limitation.

vicidial.org

wait_timeout in my.cnf & AST_manager_kill_hung_congested

wait_timeout in my.cnf & AST_manager_kill_hung_congested

Who is online