Friday, December 25, 2020

Guest Shell on CSR1000v 17.3.2

Software used:

  • GNS3 2.2.17
  • VMware Workstation for Windows 16.1.0
  • Windows 10 x64 Version 20H2 (Build 19042.685)
  • Cisco CSR 1000v (csr1000v-universalk9.17.03.02-serial.qcow2)

In VMware Virtual Network Editor, a NAT network was created with subnet address (Your subnet might be different.  But the NAT network should have been created when you install VMware).

When I created GNS3 VM, I told it to use NAT network.  As shown in the picture below, it got a DHCP IP

In GNS3, create a CSR instance and connect to GNS3-VM cloud, so that the CSR can have Internet access (to download software).  If your GNS3-VM has only one NIC, it'd be eth0.  My GNS3-VM has two NICs and the eth1 is connected to the NAT network.  When connecting CSR's Gi1 to the GNS3-VM eth1, we actually put the CSR Gi1 into the NAT network.

When CSR boots up, it shall get a DHCP IP from NAT network.  It also gets the DNS IP from DHCP.

To verify Internet is working, try to ping

Guest Shell is like a service module in the router (like the RSM in Catalyst 5500 switch, or the CUE module in Cisco 2800 router).  A Virtual PortGroup (VPG) is needed to be the gateway between Guest Shell and the physical interfaces.  In Cisco's document, VPG and Guest Shell are configured with private IP and  NAT.  See diagram below.

In home lab, you may do it in a different way.  You may configure VPG with "ip unnumber Gi1".  VGP will use the IP of Gi1.  Then configure the Guest Shell interface in the same subnet as Gi1 (but a different IP).  The advantage is - one less subnet in the network.  No NAT is needed.  The disadvantage is - you need to allocate an IP in the same subnet as Gi1. (this shouldn't be a problem in home lab though).  This option is illustrated in the diagram below:

Enable IOX:

Configure VirtualPortGroup0:
interface VirtualPortGroup0
 ip unnumbered GigabitEthernet1

Check the VPG IP:

Configure Guest Shell parameters.  In the example below, is the VPG IP. is an arbitrary (available) IP in the same subnet. is a DNS server.
app-hosting appid guestshell
 app-vnic gateway0 virtualportgroup 0 guest-interface 0
  guest-ipaddress netmask
 app-default-gateway guest-interface 0
 app-resource profile custom
  cpu 1500
  memory 512

VERY IMPORTANT: the router needs to know how to send the traffic to guest shell:
ip route VirtualPortGroup 0

Enable Guest Shell:

Enter Guest Shell.  Optionally, sudo:
[guestshell@guestshell ~]$ 
[guestshell@guestshell ~]$ sudo su -
Last login: Fri Dec 25 20:45:49 UTC 2020 on pts/4
[root@guestshell ~]# 

DNS resolution within Guest Shell is independent of host platform itself. The name-server configured in "Guest Shell parameters" will automatically get injected into the /etc/resolv.conf file on the CSR1000v. For NX-OS you must explicitly configure the /etc/resolv.conf entry.

[root@guestshell ~]# cat /etc/resolv.conf

Verify Guest Shell can ping Internet host by DNS name.

Check versions:
[root@guestshell ~]# cat /etc/*-release
CentOS Linux release 8.1.1911 (Core) 
NAME="CentOS Linux"
VERSION="8 (Core)"
ID_LIKE="rhel fedora"
PRETTY_NAME="CentOS Linux 8 (Core)"


CentOS Linux release 8.1.1911 (Core) 
CentOS Linux release 8.1.1911 (Core) 
[root@guestshell ~]#             
[root@guestshell ~]# hostnamectl
   Static hostname: guestshell
         Icon name: computer-container
           Chassis: container
        Machine ID: d1eabe2de31449ccbbc0bae3567b0b83
           Boot ID: 222a6b054eda4e3f8bb93705a9bb7a44
    Virtualization: lxc-libvirt
  Operating System: CentOS Linux 8 (Core)
       CPE OS Name: cpe:/o:centos:centos:8
            Kernel: Linux 4.19.106
      Architecture: x86-64
[root@guestshell ~]# 
[root@guestshell ~]# uname -a
Linux guestshell 4.19.106 #1 SMP Fri Oct 2 17:55:01 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[root@guestshell ~]# 
[root@guestshell ~]# uname -mrs
Linux 4.19.106 x86_64
[root@guestshell ~]# 
[root@guestshell ~]# cat /proc/version
Linux version 4.19.106 (oe-user@oe-host) (gcc version 8.2.0 (GCC)) #1 SMP Fri Oct 2 17:55:01 UTC 2020
[root@guestshell ~]# 

[root@guestshell ~]# python3
Python 3.6.8 (default, Nov 21 2019, 19:31:34) 
[GCC 8.3.1 20190507 (Red Hat 8.3.1-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.


Sunday, October 18, 2020

CUCM Security Tips

CUCM Security Tips

  • Secure Conference Bridge has special requirement.  Though configuration for secure conference bridge (CFB) is pretty much the same as secure transcoder(TRA) and secure media termination point(MTP), secure CFB requires the register name to match the hostname portion of the subject name (in the certificate/trust-point).  For instance,
    • You router's FQDN is
    • Most likely you would enroll a certificate for the router with a subject name "".
    • On the router, you created a trust point R1-Cert to hold the above certificate.
    • You would think you could use R1-Cert for secure SIP-trunk, secure media resource, etc.  Until it comes to the point that you try to register a secure conference bridge.
      • On CUCM, you saw the conference bridge status is "rejected", yet secure transcoder and MTP are registered.  So it's unlikely certificate problem.
      • On CUCM, you named them R1-CFB, R1-TRA, and R1-MTP

The problem is with the name "R1-CFB".  It has to match with the hostname portion in the certificate (which is "R1" in this case).  OK, no big deal, I'll just change the name from "R1-CFB" to "R1".  Well... you cannot do that.  Because IOS won't accept a conference bridge name shorter than 6 characters or longer than 15 characters.  So you either change the router name (make it between 6 and 15 characters), re-enroll the certificate; or keep the router name/keep the router certificate, and enroll another certificate for conference bridge (yes, for conference bridge only).  For instance,

    • Your router certificate is
    • Your conference bridge certificate is

No, you don't need DNS entry created for those names.  You may use IP addresses and TLS handshake will still succeed.

  • On IOS-XE (all ISR G3 / ISR 4000 series), Cisco decided not to support secure transcoding, which means, when you create transcoding profile on the router, there's no "security" option available.  Available options are:
    • Register non-secure transacoder to CUCM (application sccp)
    • Register LTI transcoder to the router (application cube)
    • No transcoding at all.  on IOS-XE, SRTP to RTP internetworking does not require transcoding.
  • Crypto Mismatch in ad-hoc conference (call dropped)
31948894.000 |11:19:34.039 |SdlSig   |MXErrorReport                          |interfacesEstablished          |MediaExchange(3,100,114,571)     |AgenaInterface(3,100,11,249)     |3,100,247,81913.201^^*      |[R:N-H:0,N:2,L:0,V:0,Z:0,D:0] error=0 CallMediaFailureCause=block unencrypted  media Reason=prepareAndSendOLC - Serv Param 'Block Unencrypted Calls' is true and the call is unencrypted. Hence blocking the call.
31948895.000 |11:19:34.039 |SdlSig   |MXErrorReport                          |waitStopped                    |MediaExchange(3,100,114,571)     |AgenaInterface(3,100,11,249)     |3,100,247,81913.201^^*      |[R:N-H:0,N:4,L:0,V:0,Z:0,D:0] error=0 CallMediaFailureCause=block unencrypted  media Reason=openOutgoingAudioChannel - Serv Param 'Block Unencrypted Calls' is true and sRTP keys not generated successfully. Hence blocking the call.

    The above messages indicate there's a crypto mismatch between the conference resource and the intended call leg.  Use "show sccp" command to see support crypto on conference resource.  e.g.:

    Router#show sccp
    Conferencing Oper State: ACTIVE - Cause Code: NONE
    Active Call Manager:, Port Number: 2443
    TCP Link Status: CONNECTED, Profile Identifier: 1
    Signaling Security: ENCRYPTED TLS
    Media Security: SRTP
    Supported crypto suites: AES_CM_128_HMAC_SHA1_32, AES_CM_128_HMAC_SHA1_80

    On CUBE, create a voice class for crypto and apply it to the dial-peer:

    voice class srtp-crypto 1
     crypto 1 AES_CM_128_HMAC_SHA1_32
     crypto 2 AES_CM_128_HMAC_SHA1_80
    dial-peer voice 50 voip
     voice-class sip srtp-crypto 1

    Reference #1: 

    Reference #2: (For SCCP-based signalling, only TLS_RSA_WITH_AES_128_CBC_SHA cipher suite is supported)

    Use IOS router as CA (Certificate Authority) Server

    If you have a router handy (especially with GNS3), IOS CA is probably the most convenient way to sign certificates.  Though you may set up IOS CA for online enrollment, I strong recommend you practice terminal method.  In most cases, terminal is the quickest (and probably the only) way to get your job done.

    To set up CA server on IOS, you just need a couple commands:

    conf t
    crypto key generate rsa general-keys exportable label myCA modulus 2048
    crypto key export rsa myCA pem url nvram: des myCAkey
    ip http server
    crypto pki server myDB
     database level minimum
     database url nvram:
     issuer-name cn=myCA, l=Dallas, c=US
     lifetime certificate 7305
     grant auto

     no shut

    From a router (say, R1) you want to enroll certificate, do the following:

    crypto key generate rsa label MyRSAkey exportable modulus 2048
    crypto pki trustpoint R1-Cert
     serial-number none
     fqdn none
     ip-address none
     revocation-check none
     rsakeypair MyRSAkey
     enrollment terminal
    crypto pki enroll R1-Cert

    R1 will generate CSR and print it on the terminal.  You're going to copy/paste the CSR to the CA server we created above.  An example output is like below:

    R1(config)#crypto pki enroll R1-cert
    % Start certificate enrollment .. 
    % The subject name in the certificate will include:
    % The fully-qualified domain name will not be included in the certificate
    Display Certificate Request to terminal? [yes/no]: yes
    Certificate Request follows: 
    ---End - This line not part of the certificate request--- 
    Redisplay enrollment request? [yes/no]: no

    On CA server, use global command "crypto pki server myDB request PKCS10 terminal" to sign a CSR.  An example output is like below:

    CA#crypto pki server myDB request PKCS10 terminal
    PKCS10 request in base64 or pem 
    % Enter Base64 encoded or PEM formatted PKCS10 enrollment request.
    % End with a blank line or "quit" on a line by itself.
    % Granted certificate:

    The yellow portion above is our input.  The green portion the the signed certificate (for R1).  We're going to copy/paste it to R1.  This is called "import" a signed certificate into R1.  But before importing a signed certificate, we need to import the signer (CA) certificate first.  On CA, use config command "crypto pki export myDB pem terminal" to export CA certificate.  An example is as below:

    CA(config)#crypto pki export myDB pem terminal
    % The specified trustpoint is not enrolled (myDB).
    % Only export the CA certificate in PEM format.
    % CA certificate:
    -----END CERTIFICATE----- 

    Now on router R1, do the following:

    1. Import CA certificate with config command "crypto pki authenticate R1-Cert".  R1-Cert is the trust point name, which is just a placeholder for a cert and its signer.  Paste the CA cert (the 2nd green block above).
    2. Import R1 cert with config command "crypto pki import R1-Cert cert".  Paste the signed R1 cert (the 1st green block above).

    Now you have successfully installed certificate on R1.

    If you got "Authentication failed - could not validate certificate" when trying to install the R1 certificate, it is most likely you have Revocation-Check enabled at the R1-cert trust-point.

    Instead of:

    crypto pki trustpoint R1-Cert
      revocation-check none

    You might have:

    crypto pki trustpoint R1-Cert
      revocation-check crl

    To fix the problem, set the revocation-check to none.  Then you'll have to re-generate the CSR and have it signed again by CA.

    Wednesday, October 14, 2020

    Revisit "Urgent Priority"

    Cisco CUCM (CallManager) has "urgent priority" option for translation patterns and route patterns.  At the first glance, it is pretty straight forward.  It is usually used with emergency patterns like "911".  The purpose is to eliminate potential inter-digit timeout.  For instance, when user dialed 911, CUCM will route the call immediately, even if there are potential matches (like 911XXX).

    But what happens if we use urgent priority on variable length pattern (like "!")?  Since wildcard ! means "one or more digits", shouldn't the system wait for more digit anyway?  Would interdigit timeout happen or not?

    The short answer is "No".  If you have urgent priority on pattern "!", and you are dialing digit by digit, the system will start routing the call after the first digit is pressed.  Because that matches the definition "one or more digits".  That seems pretty useless.  Why would people do that?

    It is not totally useless.  You may still dial multiple digits with the following options:

    Option 1: Bloc-Dial

    Keep the phone on hook (do not get a dial tone), enter all the digits you want, then hit the "Dial" button.  This is called bloc-dial.  You may pass all digits to ! pattern with urgent priority with bloc-dial.

    Option 2: From previous hop

    In large-scale dial plan design, we usually expose translation patterns(TPs) to phones, but not route patterns(RPs).  The intend is to use TPs to do all kinds of digit manipulation and class of control.  Then pass the manipulated digits to RPs.  In this case, TP has no problem passing all digits to RP (even if the RP has urgent priority).

    Let say, you have a two-tier dial plan design (TP/RP).  You have emergency TPs with urgent priority.  When those TPs pass digits to RPs, it may or may not experience interdigit timeout depending on your RP setup.  Interdigit timeout is evaluated at each hop.  If you use RP ! to catch all digits passed by TPs, you might want to enable urgent priority on that ! pattern.  Or as an alternative, you may enable "Do Not Wait for Interdigit Timeout On Subsequent Hops" on the TP.

    Another interesting topic is the interaction of urgent priority and "longest match".  Take a look at the following patterns:

    • 7XXX
    • 700XX

    When you dial 7, 0, 0, 1, 2 digit-by-digit, which one will be matched?  At the first glance, 700XX seems to be the best candidate because it matches more digits.  Enable urgent priority on 7XXX seems harmless for this dialing string, right?  Actually not.  When urgent priority is enabled on 7XXX, you won't be able to enter the fifth digit.  Once you entered the 4th digit, system immediately routes the calls.  The only way to work around that is to use bloc-dialing.

    In summary:

    1. Interdigit timeout applies to digit-by-digit dialing.  It does not apply to bloc-dialing.
    2. Digits passed by previous hop does NOT equal to bloc-dialing.  For instance, urgent priority on TP level does not necessarily immune to interdigit timeout at RP level.
    3. Be careful of overlapped patterns.  Using urgent priority might have side effects.

    Sunday, May 24, 2020

    cEdge - Single Image Switch to Controller Mode

    cEdge is a Cisco router acting as SD WAN Edge.  Cisco's road map is to replace vEdge products with cEdge.  cEdge can be on ISR, ASR and CSR, as well as their corresponding virtual variants (such as ISR 1000v, CSR 1000v, etc.)

    When the above devices operating in "regular mode", it's called "Autonomous Mode", which is your good old IOS XE command system.  When they operate in "SD WAN mode", it's called "Controller Mode", which is Viptela-like command system.

    In pre-17.2 versions, you'll have to load different software on the router to support different modes.  Since version 17.2.1r, one single image can support two different modes.  Please see

    I put CSR 1000v 17.2.1r in my GNS3 lab.  It boots into autonomous mode by default.  I tried to switch it to controller mode with no luck.  I was referring to  Also

    I did quite a lot research.  However, this version is too new to yield any helpful resource online.  I finally figured it out.  It's a documentation issue.

    First, let's take a look at Cisco documentation below:

    Per above documentation, the cfg file (in router bootflash:) will trigger the mode change.  That is NOT TRUE!  At least not in CRS 1000v 17.2.1r.  The cfg file won't be used UNTIL the router is switched to controller mode.

    To switch from autonomous mode to controller mode, you use CLI command "controller-mode enable".  Router will warn you that all configuration will be lost.  After conformation, router will reboot into controller mode.

    The first sign of controller mode is - you'll be prompted to enter username/password, even if you don't have it set up previously.  Default username/password is admin/admin, which aligns with Viptela defaults.

    After login, you may use the following commands to double confirm it's in controller mode: