Thursday, June 18, 2015

Cisco IOS AAA Configuration with ISE

I was trying to set up a POC (Proof Of Concept) lab to use Cisco ISE as AAA server.

IMHO, a good configuration example should meet the following:

1) Minimal
No one wants to read a 300-line example.  A minimal configuration is easy to understand.  If needed, we may add features on top of the minimal configuration.

2) Typical
The example should represent the most popular use case.

3) No Ambiguity
Though not affecting functionality, names should be picked carefully to avoid any potential confusion for readers (especially beginners).

My goal was to set up AAA on a Cisco router with Cisco ISE for IOS CLI.  Based on the username, IOS privilege level 7 or level 15 will be assigned after login.

This is a typical use case as RBAC (Role Based Access Control) is widely used.  Users with privilege 7 can run most of the "show" commands but not the "conf t" command.  Users with privilege 15 can run all commands.

I have ISE 1.4 and UNL 0.9.0-40 running on VMware Workstation.  The topology is like below:

ISE IP = 192.168.21.131
Router IP = 192.168.21.201



Router configuration:

! -- encrypt passwords in config file
service password-encryption
!
! -- fallback user account in the event of RADIUS failing
username admin password cisco
enable password cisco
!
! -- define RADIUS server
radius server RADIUS-ISE
 address ipv4 192.168.21.131
 key secret123
!
! -- create AAA server group
aaa group server radius RADIUS-ISE-GROUP
 server name RADIUS-ISE
!
! -- creaate new AAA model
aaa new-model
!
! -- Configure the default login method to 'none' (no authentication)
! -- This method applies to the console by default
aaa authentication login default none
aaa authorization exec default none
!
! -- Configure Authentication and Authorization methods for VTY lines
aaa authentication login VTY_authen group RADIUS-ISE-GROUP local
aaa authorization exec VTY_author group RADIUS-ISE-GROUP local
!
! -- Configure default Accounting method, which applies to both console and VTY lines
aaa accounting exec default start-stop group RADIUS-ISE-GROUP
!
! -- Apply Authentication and Authorization methods to VTY lines
line vty 0 4
 authorization exec VTY_author
 login authentication VTY_authen
 transport input all



ISE configuration:


1.     Administration > Network Resources > Network Devices > Default Device


For simplicity, we use “Default Device” here.  You may create device group to have more granular control.

Enable “Default Network Device Status”.
Enter “secret123” into “Shared Secret” field.

2.     Policy > Policy Elements - Results > Authorization > Authorization Profiles

 Create two authorization profiles:
  • Shell_priv_15
  • Shell_priv_7

Attribute settings for profile “Shell_priv_15”:
Cisco:cisco-av-pair
Shell:priv-lvl=15
Radius:Service-Type
Login

Attribute settings for profile “Shell_priv_7”:
Cisco:cisco-av-pair
Shell:priv-lvl=7
Radius:Service-Type
Login


3.     Administration > Identity Management > Groups

Create two User Identify groups:
  • CLI-users-15
  • CLI-users-7

4.     Administration > Identity Management - Identities > Users

Create users and put into desired group

 

5.     Policy > Authorization

 Create two authorization policies on the top:
  • CLI-access-15
  • CLI-access-7

Associate corresponding user group to “Conditions” column.
Associate corresponding authorization (standard) profile to “Permissions” column.





Monday, May 25, 2015

New book - Deploying Cisco Unified Contact Center Express

Back in 2006 I was a Cisco TAC engineer supporting Contact Center Express (a.k.a. IPCC Express, CRA, CRS, IPIVR, now known as CCX).  Information is so scarce that even TAC engineers joked it was "tribal knowledge".

To make things more interesting (instead of saying "worse"), CCX components were developed by different companies - CCX Engine was developed by Cisco while Desktop Suite was developed by Spanlink (now Calabrio).

Cisco's documentation is "menu driven" instead of "task driven".  i.e. the documentation explains what each menu does but doesn't tell you what menus you need to navigate to achieve a specific goal.

Training partner classes are more focusing on scripting other than system integration.

Since then I was hoping to write a book on CCX.  Not a "CCX For Dummies" but a book for engineers.  For engineers that are smart enough to grasp any new products or technologies with a lab and a good book.

Different people have different definitions of "good book".  My definition is:

1) KISS - Keep It Simple Stupid.
2) Tell me something new - Something that the manufacturer manual didn't tell me.

I've been procrastinating due to work and family.  Now it's 2015.  I finally had the chance to finish something I wanted to do nine years ago.  For the time being, the book is only available from LuLu.com (http://www.lulu.com/shop/michael-houtong-luo/deploying-cisco-unified-contact-center-express/paperback/product-22186750.html).  It will be available on other retailers (such as Amazon, Barnes & Noble, etc.) later on.  Table of content can be viewed from here.

Same as my other book "Deploying Cisco Unified Presence", I did this with limited time and resource.  I'm pretty sure there will be typos and mistakes in the book.  I'd appreciate it if you can send me your feedback and suggestions to houtong@gmail.com.

By the way, I'm setting up a charitable foundation (Luo's Foundation) to support  students from low-income families.  All proceeds from this book will go to the foundation.  I will publish the details when the foundation is fully set up.

Thank you very much for your support.

http://www.lulu.com/shop/michael-houtong-luo/deploying-cisco-unified-contact-center-express/paperback/product-22186750.html


Tuesday, May 5, 2015

ESXi 6.0 on HP DL380 G5

Just upgraded ESXi from 5.5 to 6.0 on my HP DL380 G5 with the "HP Custom Image".  Thought it'd work better than the "vanilla" (generic) image.

The install went well.  But from time to time, it kept giving me the purple screen (system crash), usually 10 - 20 minutes after the VMs boot up.

A search led me to the URL http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2085921

But it didn't fix my problem.

Finally, I decided to "upgrade" with a generic image - overwrite the HP custom version with a generic version.

The installer gave me warning on the RAID controller.  P400 controller was "not supported" by 6.0 any more.  I ignored and continue.  The install completed successfully.

Now I'm living happily ever after.  :)

Saturday, April 4, 2015

IOU, GNS3, Cisco VIRL/CML

GNS3 is a well known network emulator software.  Many people compare it with IOU, which is not quite an apple-to-apple compare.

GNS3 is actually a front end add-on to DynamIP - which is the real emulator. I'm not going to compare DynamIP and IOU here as this has been done many times.  You may search Internet for those articles.

DynamIP and IOU requires quite some command line effort to get them running.  Thus some "front end" add-ons were built to make them easier to use.  GNS3 is the most popular front end of DynamIP.  On IOU side, it's probably IOU Web.  These add-ons allow you crate network topologies, save/restore configurations, etc.



From easy-of-use perspective, GNS3 is no doubt a winner.  You may drag-and-drop routers/switches in GNS3.  However, I prefer IOU because it uses much less resource (CPU/memory) and I'm not afraid of writing up the NETMAP file.

IOU Web doesn't have drag-and-drop.  But it has quite a few neat features.  For example, you may use any existing network diagram and map the elements on it to any virtual devices.  It also has a very powerful configuration/lab management interface.

Things become interesting now when GNS3 can act as IOU front-end.


I tested GNS3 version 1.3 with IOU back end and it works pretty well.  I think here's how I would choose between GNS3 and IOU Web:

If I need to create a lab in a very short time (say 20 routers in 15 minutes), I'd use GNS3.  Otherwise I'd probably use IOU Web as it gives me more flexibility on documentation and network drawings.  Also, as a "minimalist", IOU Web doesn't require any front end software (as it has a http server built-in and uses web interface).

How about Cisco VIRL/CML?

Sigh...  when I first heard about the idea two years ago, I was so excited.  Finally, an official / vendor-supported network emulator!  With the progress lagging on, we're getting more and more frustrated and disappointed.

Yes, Cisco VIRL/CML does have some unique features such as auto-generated configuration, TAC support, etc.  But aren't those features more "novice facing"?  The most disappointing part was - Cisco didn't (and has no plan) to invest more on the L2 features.

I had a conversation with the CML product manager lately.  He admitted that the NX-OS feature on CML was "70% done" because quite a few developers in the Nexus team were laid off.  Thus the most interested features such as VDC, VPC are not available on CML.  I asked if there's ANY possibility that Cisco continues the development and complete the rest 30% in the future, his answer was a very definite "No".  Then I asked him what's the selling point of CML?  He just kept repeating "more than 15 nodes" and "TAC supported".

VIRL (personal edition) is selling for $200.  CML (enterprise edition)'s price structure is unknown.  But price is not the point here.  Even if they are free, what's the advantage they have over GNS3 and IOU?

Friday, October 24, 2014

ESXi 5.5 support for CSR 10.5

I've been testing CSR 10.5 (UCM 10.5, UCCX 10.5) on ESXi 5.5 U2.

I kept running into problems I've never seen before, such as the lovely VMware "pink screen" (Pink Screen Of Dead).  Consistent high CPU usage on UCM and UCCX, etc.

I couldn't find the pattern of "pink screen".  But it happens quite a couple time, which I've never seen in the past 8 years with Cisco UC on VMware.

The other issue is high CPU on UCM and UCCX.  The CPU usage was consistently at 99% that I couldn't even log into CLI.

Just FYI that that server is a HP DL380 G5 with 32G of RAM and 8x146G SAS drives (a.k.a. Cisco MCS-7845-H2).

Did some research online.  The culprit seems to be the Cisco OVA file.  Somehow the OVA works fine on ESXi 5.0 but not ESXi 5.5.  The solution is NOT to use the OVA.  Instead of importing the OVA, I manually created the virtual machine with the specification in OVA.

CUCM:
1 CPU 1 Core
4G RAM
80G HDD
LSI Logic Parallel
VMXNET 3 NIC

UCCX:
1 CPU 2 Core
8G RAM
146G HDD
LSI Logic Parallel
VMXNET 3 NIC

Things seem to be much better now.  CPU is around 10%.  Maybe it's just me.  But if you're running into the same issue, it's worth trying.  You don't have to reinstall the VM.  Just create a VM from scratch (without using OVA).  Then point the hard drive to the existing VM.

This is CPU utilization with fresh created OVA.  After the system "warmed up" (about 30 minutes), utilization drops from 99% to 10%.



Some reference links:
http://ciscocollab.wordpress.com/2014/01/28/esxi-5-5-support-latest-information/

http://docwiki.cisco.com/wiki/Unified_Communications_in_a_Virtualized_Environment

https://communities.vmware.com/thread/459962



Tuesday, September 30, 2014

"Cloud" device in IOU Web

I've been using IOU Web for network emulation.

"Cloud" device is the bridge between internal devices (such as routers within IOU) and external devices (such as PCs, a real/virtual router outside of IOU, etc.).

I'm not going to get into the details of how to set up VMware network or IOU.  There are plenty of documents online about that.

What I'm going to share is the solution to a weird problem.

I wanted to build a simple lab as shown below.  Two LAN segments are connected via two routers back-to-back.


NETMAP file and device config as below.


Pretty straight forward, right?  But the problem is - I cannot turn on device 1 (LAN1).  Notice that device "LAN1" stays in red below which means it's off.

I scratched my head for quite a while.  Tried to tweak the parameters, device ID, naming, IOU host, VMware Network Editor.  No avail.

Then I looked at the logs and noticed the following:

Why it asked me check the NETMAP file?  I don't see any error there.  What is "instance"?  Why is it not found?

After a little bit research, I realized "instance" is the same as "device".  As shown in the diagram above, we have four instances - 1, 2, 3 and 4.

We have problem with instance 1 (LAN1), which is connecting (referencing) instance 2 (R1).  If the system was complaining about "instance not found", it can only be 1 or 2.

I also noticed that instance 4 (LAN2) always works.  What's the difference between 1 and 4?

It turns out that in NETMAP file (connection definition), the "cloud" device cannot be the preceding one.  The "correct" NETMAP should be written like this:

 Notice that instead of "1:0/0 2:0/0", I swap them and make it "2:0/0 1:0/0".  Then try to start the LAN1 device.  There we go:

This seems to be a software bug.  But the point is - a good engineer should be able to recognize the pattern from the symptom, perform deductive reasoning, and propose possible solution.  :)

Tuesday, September 23, 2014

UC 10.5, ESXi 5.5U2, DL380 G5

My home lab has been collecting dust for a while.  During the weekend, I wanted to refresh it with the latest and greatest, which means:

1) Upgrade the server (MCS7845-H2 a.k.a. HP DL380 G5) BIOS and firmware.
2) Upgrade VMware ESXi 5.0 to 5.5U2.
3) Upgrade UC 7.0 to UC 10.5.

It turned out that upgrading a system that's been collecting dust is VERY different from upgrading a system that's been up and running.


First of all, the system won't boot.  Just gives me long beeps and the "Internal Health" and "External Health" LEDs are both red.  Pull all memory chips out and resit them solves the problem.

Then iLO configuration seems to be lost due low power level of the system battery.  I can't log into iLO at all (the 'default password' is system specific with unique numbers).  Set the "System Maintenance Switch" S1 to "On" bypasses the iLO password.


When trying to upgrade to ESXi 5.5 U2, I got the following error:


I know what it is.  But how could this be not enabled while I have ESXi 5.0 on it before?  Maybe it's also due to the motherboard battery?  Anyway, go into BIOS and enable the "No-Execute Memory Protection".

After ESXi upgrade, I noticed that VMware persuade move from native VM client (based on C#) to "Web Client" (based on Adobe Flash).  The initiative is to move from "fat client" to "thin client" so all new features can be hosted on the vCenter server.   You may still use the "native client" but some of the features will be missing.  Features as basic as editing a version 10 VM settings.




In order to use the "Web Client", you'll have to set up a vCenter server.  Also, to view VM console from web browser, you'll need to install a plug-in, which doesn't work with Internet Explorer (as of today).

When installing UCM 10.5, it took extremely long (> 10 hours).  Further investigation revealed that the array controller battery died.  Without battery, the array controller will disable cache, which makes it very, very slow on a RAID5 (slower than my laptop).

I have multiple options:

Option 1: Order one from eBay.  It's not expensive (~ $12 a piece).  The problem is - this kind of batteries are obsolete.  Thus the ones on eBay are all used ones, which were manufactured a couple years ago.  Who knows how long they'll last.


Option 2: Make my own battery like this: http://opensource.wrenhill.com/?p=63.  Then I can use cheap AA or AAA batteries instead of buying proprietary ones.

Neither of the above options is quick enough for me.  Thus I choose...

Option 3: "Enable Cache Without Battery".

To do this, you'll need ACU (Array Configuration Utility).  You can do it with the ROM-based interface (BIOS).

With VMware ESXi, the easiest way is to download the "offline ACU", which is a CD you boot from.  Then configure the array controller from there.


For a RAID, it's the write operation that takes more time.  Thus you want to make sure the write cache is not zero.


Last but not the least, download HP SPP DVD to update all firmwares and BIOS.

P.S. DHCP doesn't work on UCM 10.5 in case you want to use UCM as a DHCP server.  https://supportforums.cisco.com/discussion/12224526/cucm-105-dhcp-not-working