Time Sync Problem

[jcapen]

Hi all – accidental Dialer Admin here and (once again) in over my head. Hoping someone can offer advice or point me in the right direction.

The TL;DR:
Any guidance on how to troubleshoot a recurring "time synchronization" error that kicks all agents out at once?
Any recommendations for hosted Vici solutions that can support customizations and offer solid, responsive support?

The Issue:
Agents are being kicked out of the dialer with the error:

"There is a time synchronization problem with your system, please tell your administrator."

This has happened:
April 10
April 15
Twice (so far) on April 17

It’s affecting everyone logged in at once.

Our Setup:
ViciDial Version: 2.14-692a
Build: 180927-0018
Using webphone
Calls are routed through a direct circuit (not over public internet)
We are load balanced across 2 servers
Usually 40–50 agents logged in simultaneously
Campaigns are almost entirely outbound
Our office/agents are in Eastern Time, while the hosted dialer company is based in Mountain Time

Additional Notes:
We're using a third-party hosted Vici solution with some customizations layered in. Unfortunately, support hasn’t been very helpful so far. The advice of this forum has been helpful in the past, so I’m hoping someone here can at least help me ask the right questions.

I’ve already read through other “Time Sync” threads, but the variety of solutions makes it tough to narrow down what might be relevant for our specific setup.

Thanks in advance for any help or recommendations!

[williamconley]

Good job posting your Vicidial Version with Build. You should also add your installer with full version if available (some of this often shows in the splash page when logging in to the console).

Time sync is usually (75%?) a misnomer for Networking problems.

A specific field is updated every time an agent's web browser sends and receives an AJAX packet from the php page on the agent web server. Each time that packet is generated, the php page updates a TIME field on the database server specific to that agent. If that field is ever off by more than six seconds ... TIME SYNC error is tossed and the agent is kicked.

So this trigger can happen if time is not syncrhonized between the web server and the database server OR if agent packets are blocked for more than six seconds during any network hiccup.

Hope that helps.

FYI: The Vicidial Group offers hosting for Vicidial. They ARE the best most reliable hosts and they maintain the entire system including operating system and Vicidial software. If you are unhappy with your host, check them out.

Of course, PoundTeam Incorporated (who pays for me to babble on here) also provides hosting. While not as perfect as The Vicidial Group, we do hold our own fairly well IMHO.

[carpenox]

40-50 agents with what dial ratio? if you are going above 4 ratio with 50 reps, youre gonna have these type of issues

[williamconley]

40-50 agents with what dial ratio? if you are going above 4 ratio with 50 reps, youre gonna have these type of issues

I'm gonna disagree with you here. This is not a "load related" issue unless there is an underlying network-choke or underpowered DB server or a clocking issue with one of the dialers. Even our old Standard servers still handle 25 agents per server with any dial ratio that does not exceed the capacity of the dialers themselves. The problem "time sync" is not what happens, the dialers actually overload when capacity is exceeded and call quality drops. But time sync? Nah.

[carpenox]

oops youre right, my math was off, you could get about 8 ratio in with that many reps

[williamconley]

or higher if you configure the system with a max trunk setting that kicks in before your overload. then the system will dial as heavy as it can ... even when a lot of agents are on the phone.

i've had a lot of servers routinely on 10 and not max out. but i generally recommend they push as hard as they can until the (every minute) test call quality suffers, then back off 10% and set that as the trunk max. at that point, they can set the dial level anywhere they want and the max trunk will stop her from overloading. profit!

[jcapen]

Thanks everyone for the advice. Our vendor "Turned off option 'auto restart' on agent Asterisk Servers" and we have not had any time sync errors since. I'm a bit wary of what other issues this may cause but that's a problem for future me.

[williamconley]

Thanks everyone for the advice. Our vendor "Turned off option 'auto restart' on agent Asterisk Servers" and we have not had any time sync errors since.

Turning this setting off is 100% UNrelated to "time sync problem"

I'm a bit wary of what other issues this may cause but that's a problem for future me.

And you should be. This feature will restart asterisk if it dies. Without this feature, if asterisk crashes ... instead of having all calls drop and then having the system back online in about a minute, asterisk will stay dead. Not a happy moment. So they changed something else or their system started working coincidentally when they made this (apparently randomly chosen) change.

OR (and work with me here), they made that up to cover the fact that they screwed something up and then fixed it. But certainly had no interest in tell you what they screwed up.

Consider looking at new hosting/leasing options. I hear Vicihost is adequate (by which I mean they've been running their hosting system forever and it's the best place to have SOLID Vicidial servers running on the planet). I'm kind of partial to PoundTeam's colo myself (since I work here). But don't let that go until your present host screws up something else. LOL (be prepped)