Cisco Voice Guru

Most of the content from this troubleshooting post was gathered from the H.323 Gatekeeper Troubleshooting document on Cisco’s docwiki site.  I have tried to summarize the important points below.  Thanks to Zen Beck for pointing me in the right direction.

Troubleshooting H.323 Gatekeeper Registration

To determine registration reject issues, run a “debug h225 asn1” and search for “rejectReason.”  Below is a list of the registration reject reasons and corresponding explanations:

RRJ: rejectReason duplicateAlias

This is usually the result of the gateway registering a duplicate of an E164-ID or H323-ID-another gateway has already been registered to the gatekeeper. If it is a duplicated E164-ID, change the destination pattern configured under a POTS dial-peer associated with an FXS port. If it is a duplicated H323-ID, change the gateway’s H.323 ID under the H.323 VoIP interface.

RRJ: rejectReason terminalExcluded

This is the result of the gateway subnet being disabled in the gatekeeper. Check the gatekeeper configuration. Removing the “no zone subnet gk” command resolves the issue. To remove the command completely, remove the “zone local gk” command.

RRJ: rejectReason securityDenial

This RRJ is the result of the security commands being enabled in the gatekeeper, and the inability of the gateway to match the H.323-ID, E.164-ID, passwords, or security token that the gatekeeper requires. To resolve the issue, check the securiy configuration in the gatekeeper.

RRJ: rejectReason invalidAlias

The RRJ is the result of a no-zone prefix defined in the gatekeeper. Check the configuration on the gatekeeper and add the zone prefix with the proper E.164 address.

Troubleshooting H.323 Gatekeeper Call Routing and Dial Peers

The following is a list of useful show and debug commands used to verify and troubleshoot gatekeeper and gateway call routing issues.

• show gateway - Used to verify E.164 and H.323 alias registration for the gateway
• show gatekeeper endpoints – Used to verify the E.164 and H.323 alias registered with the gatekeeper
• show gatekeeper gw-type-prefix - Used to verify E.164 prefix registrations on the gatekeeper
• show gatekeeper zone prefix | status - Used to verify zone status and configuration parameters
• debug ras - Applicable for gateways and gatekeepers
• debug debug h225 asn1 - Applicable for gateways and gatekeepers
• show dial-peer voice - Used to verify configured technology prefixes under the dial-peers

Troubleshooting H.323 Gatekeeper Bandwidth

When deciding whether there is enough bandwidth to accept a call Admission Request (ARQ), the Cisco gatekeeper calculates the available bandwidth with the following formula:

Available bandwidth = (total allocated bandwidth) – (bandwidth used locally) – (bandwidth used by all alternates).

If the available bandwidth is sufficient for the call, an Admission Confirmation (ACF) is returned, otherwise an Admission Rejection (ARJ) is returned.

Three show commands give you visibility into the bandwidth management on the gatekeeper:

• show gatekeeper zone status
• show gatekeeper zone cluster
• show gatekeeper calls

Troubleshooting Gatekeeper Endpoint Call Admission Issues

Admission Confirmed (Busy Tone Back)

If the Admission Confirmed (ACF) message has been sent by the gatekeeper and arrived at the endpoint side, but the call still received a busy signal, check to see if the terminating IP address in the ACF is an expected valid endpoint IP.  If the ACF has an IP address of the terminating endpoint, remove the gatekeeper and make a direct endpoint-to-endpoint call to see if a call can be established.

Admission Reject (ARJ) rejectReason calledPartyNotRegistered

This is a common reason for rejection. It is captured from the local or originating gatekeeper when the gatekeeper has no information on where the called number should be terminated. There are two ways this problem can occur. One reason is that the call terminates at a gateway, and the gateway is not registered with the E.164 address or with a tech-prefix. To resolve this, make sure the gateway is registered with a tech-prefix to the gatekeeper.

A second possible explanation for this error message arises if the called party is a terminal in a remote zone. It might be that the terminal does not have a proxy enabled in the same gatekeeper zone where it is registered.

Note: Intrazone calls do not require the match of a zone prefix.

ARJ "rejectReason requestDenied"

The rejection shown here comes about because the endpoint-requested bandwidth exceeds the limit configured in the gatekeeper. To resolve this, increase the bandwidth in the gatekeeper using the “bandwidth” command under the gatekeeper mode, or lower the bandwidth request from the endpoint.

Refer to the “debug h225 asn1” for the requested bandwidth in the ARQ (Search for “admissionRequest” in the otuput).  Compare requested value to the “gatekeeper zone status” to see if enough bandwidth is available.

TCP Port Numbers

• TCP 1720 – H.225 Call Setup / FastStart
• TCP 2000 – SCCP
• TCP 2428 – MGCP ISDN PRI Backhaul
• TCP 5060 – SIP Telephone
• TCP 11000-11999 – H.245 Capabilities Exchange / SlowStart

UDP Port Numbers

• UDP 1718 – H.225 RAS Multicast Gatekeeper Discovery
• UDP 1719 – H.225 Gatekeeper RAS Messages
• UDP 2427 – MGCP Control
• UDP 2748 – CTI/JTAPI
• UDP 5060 – SIP Trunk
• UDP 16384 – 32766 (even port numbers) – RTP
• UDP 16385 – 32767 (odd port numbers) – RTCP

CRCX

I’m on a roll tonight.  Here are some more troubleshooting/FYI notes, this time focused on SRST.  The study life of a dedicated CCIE candidate is one constant stream of information absorption occasionally interrupted by family time, eating, sleeping, and relieving oneself in the bathroom.  Sometimes the latter gets delayed by long chapters on gatekeeper behavior (as I’m experiencing right now!).

DLCX

Be aware of the following SRST restrictions:

1. Extension mobility does not work in SRST mode when using call-manager-fallback.  CME as SRST does have extension mobility, but login information is not seamlessly transferred during failure situations.
2. SRST does not have a concept of partitions or calling search spaces. CoR can be used to some extent, but it is a weak-comparison and is not automatically populated on the SRST router.
3. MWIs might be out of synch when recovering from SRST mode.
4. All forwarding (busy, no answer, and forward all) is lost in SRST mode.
5. In SRST mode, all call routing decisions are made on the SRST gateway through dial peer configuration, so anything not explicitly configured on the SRST gateway will not work.

For SRST mode, be sure to configure voicemail under “telephony-service” and to configure an “ephone-dn template” to set the necessary call-forward noan and busy behaviors.

Set your DHCP lease times to several days, or use a local DHCP server to ensure that IP Phones do not reset themselves when their lease expires during a WAN outage.

Use transfer patterns to allow transfers to non-IP Phone destinations.

Survivable endpoints: IP Phones and MGCP Gateways

Nonsurvivable endpoints: H.323 Gateways, SCCP Gateways, CTI/TAPI Endpoints

The rules for call survivability can be summarized as follows:

• If the call involves nonsurvivable endpoints, and a CallManager involved in the call fails, the call fails.
• If the call involves one nonsurvivable endpoint and one survivable endpoint, the call fails only if the CallManager that the nonsurvivable endpoint is registered to fails.
• If the call involves only survivable endpoints, and one or more CallManagers involved in the call fails, the streaming connection between the endpoints is maintained. Note, however, that the endpoints do not have supplementary services available to them after the failure.
• If a hardware-based conference bridge is involved in a call, and the CallManager controlling the conference bridge fails, all calls from nonsurvivable endpoints fail. All calls from survivable endpoints continue in the conference.

  If a software-based conference bridge is involved in a call and the CallManager controlling the conference bridge fails, all calls from the nonsurvivable endpoints fail. All calls from survivable endpoints continue in the conference until the party terminates the call voluntarily. The conference bridge reregisters with an available CallManager.

• If the call involves an MTP or a transcoder, the call fails.
• If the call is in the and a CallManager involved in the call fails, the call fails.

The IP Voice Media Streaming
Application supports only G.711 µ-law and G.711 A-law codecs. It can provide up to 128 streams of conferencing on a standalone server or 24 streams when it coresides with CallManager on the same server.onf

Conference Bridges, Transcoders, and MTPs: Best Practices

• When a media resource is assigned to any MRG, it is no longer available as part of the default MRGL. This means that endpoints that require the use of a media resource in an MRG must have an MRGL configured that contains that MRG.
• Be extremely careful when configuring your codec selections between regions. This is by far the most common cause of transcoder problems.

Troubleshooting MOH

1. Check the MOH server registration status.
2. Check the MRG and MRGL configuration.
3. Verify router configuration for multicast (if multicast is used).
   ip multicast-routing
   ip pim sparse-mode | ip pim dense-mode
4. Verify the multicast capability of the terminating voice gateway.
5. Verify the codec used by all devices involved.
6. Verify the location’s configuration if you are using the centralized call processing model.

H.225/H.245 Summary

H.225 RAS Signaling:

• RAS is the signaling protocol used between gateways and gatekeepers. The RAS channel is opened before any other channel and is independent of the call setup and media transport channels.
• RAS uses User Datagram Protocol UDP 1719 (H.225 RAS messages) and UDP 1718 (multicast gatekeeper discovery).

H.225 Call Control (Setup) Signaling

• H.225 call control signaling is used to setup connections between H.323 endpoints.
• A reliable (TCP) call control channel is created across an IP network on TCP 1720. This port initiates the Q.931 call control messages for the purpose of the connection, maintenance, and disconnection of calls.

H.245 Media Control and Transport

• H.245 handles end-to-end control messages between H.323 entities.
• H.245 procedures establish logical channels for transmission of control channel information.
• It is used to negotiate channel usage and capabilities such as flow control and capabilities exchange messages.

Outside Dial Tone Played at the Wrong Time

• If multiple patterns exist with the same first digit and at least one of the patterns is not configured to provide outside dial tone, outside dial tone is not played unless all the remaining potential matches are configured to provide it.
• Fix 1: Check the box on the 911 route pattern that tells it to play outside dial tone.
• Fix 2: Change the 911 pattern to 9.911 instead and check the box that tells it to play outside
  dial tone (although 9.911 is already included as part of 9.@).
• A good way to isolate the problem is to view the Route Plan Report in Cisco CallManager Administration (Route Plan > Route Plan Report).

Call Forward All (CFA)

• For calls within a cluster, use the Forward Maximum Hop Count service parameter (Service > Service Parameters > select a server > Cisco CallManager) to end a forwarding chain after a specified number of hops
• Use MaxForwardsToDn and StopRoutingOnUserBusyFlag to prevent routing loops that traverse other clusters or legacy TDM equipment.

Called and Calling Party Transformations

1. The External Phone Number Mask on the original calling device (when specified on a route pattern or translation pattern by the Use Calling Party’s External Phone Number Mask checkbox.)
2. A translation pattern
3. A route pattern
4. A route list using route group details
5. Transformation on the terminating device

Methodology for Resolving Call Routing Problems

1. What calling device is having the problem (phone, gateway, CTI ports, Voicemail ports, etc.)?
2. What is the dialed number as dialed by the user or sent by the gateway?
3. What calling search space (and corresponding partitions) are being searched?  Pay attention to the different line/device CSSs that come into play.
4. If a match is made (important word = IF), what pattern is matched?  What partition does that pattern belong to?
5. Is a translation pattern being us?  If so, where is it redirecting the call?
6. Are there any DDIs taking place?
7. If the route pattern goes to a route list (i.e. 99% of patterns), check the route list for digit manipulation being done at that level.
8. If you’re sure the outbound call is hitting the gateway, use my best friends:
   debug isdn q931
   debug voip dialpeer (H.323/SIP only)
   debug voice translations (H.323/SIP only)

Random Notes

• The acronym DDI refers to “digit discard instruction” and not “direct dial in” which is commonly used to refer to direct inward dial (DID) numbers in some countries.