Monday, March 1, 2010

Calendar presence with CUPS and Exchange

Cisco Unified Presence Server (CUPS) has the capability to retrieve calendar (Exchange) status and populate to clients (CUPC). When your calendar status is "busy" in calendar, CUPC will display your status as "in a meeting".



To enable this feature, there are server-side configuration (CUPS/Exchange) and client side configuration(CUPC).

CUPC configuration

On CUPC side, you go to "File > Preferences > Status" and put a check mark on "Show me as 'In a Meeting' whenever my Exchange calendar shows me as busy".


Please note that when multiple 'presence' arrIve, CUPC can only display one of them. The priority, from high to low is: Phone Presence > Calendar Presence > Availability Status.

For example:

You logged into CUPC ==> Availability Status = "Available"
You're on the phone ==> Phone Status = "On the Phone"
Your calendar is busy ==> Calendar Presence = "In a Meeting"

In this scenario, CUPC will display the status as "On the Phone". You won't see "In a Meeting". However, you may move the mouse over a contact (or your self-status) to see all the presence.

In the screenshot below, you'll see the CUPC display its self-status as "On the phone". When you move the mouse over the self-status, you'll see "Online, On the phone, In a meeting" in the tooltip. This is how to explore the "hidden" presence from CUPC.


CUPS configuration

The most confusing (instead of difficult) part on CUPS is the SSL/Certificate configuration. CUPS can only talk to Exchange via HTTPS. Why is that?

That is because CUPS use an Exchange "service account" to read everybody's mailbox (calendar). If the communication was not encrypted (SSL), it would pose a threat to information security. A hacker could easily intercept the packets on network and retrieve everybody's (including CEO's) email and calendar.

With that said, here are the requirements on Exchange and CUPS to set up a SSL connection:
1) HTTPS needs to be enabled for OWA (Outlook Web Access)
2) The CA certificates of Exchange need to be trusted by CUPS.

Regarding #1, there's a catch. We'll discuss that in the "Exchange" section below.

Regarding #2, there are two catches -
A) Don't confuse the CA cert with identity cert.
B) Request address has to match certificate.

A) What is a "CA cert" and what is an "identity cert"?

For example, if your Exchange OWA has a certificate to identify itself as "owa.acme.local". This certificate is called "identity cert". Let say, this identity cert was issued by a CA (Certificate Authority) called "my-ca-server.acme.local". On the CA server, it has a certificate to identify itself. We call that cert a "CA cert".

It is the "CA cert" CUPS server needs to trust. In our example above, it's the "my-ca-server.acme.local" certificate needs to be uploaded to CUPS as "Presence Engine-Trust".

B) When you configure the "Outlook presence gateway" on CUPS, the address you put in there is the address CUPS will use to sent request to Exchange.

For simplicity, quite a few people just put the IP address of Exchange there. This is not going to work with SSL. For security, the request address has to match with the common name in the identity certificate. Usually, the certificate will identity the server in FQDN (e.g. owa.apps.local). Thus, you'll have to configure the same name in presence gateway configuration page.

Please note: I used the word "same name" instead of "FQDN". Because technically, you could configure any name in the certificate. Let's take a look at a couple examples:

Your OWA server IP = 192.168.1.100
Your OWA server FQDN in DNS = owa.apps.local

Example #1: You configured the common name in the OWA cert as "email.apps.local".

This won't work. Because:

If you used "192.168.1.100" in presence gateway configuration, it doesn't match "email.apps.local" in the identify cert.

If you used "owa.apps.local" in presence gateway configuration, it doesn't match "email.apps.local" in the identify cert.

If you used "email.apps.local" in presence gateway configuration, CUPS won't be able to resolve the name, because it's not in DNS.

Or even worse, you don't have DNS configured on CUPS so you can't use any FQDN.

Solution:
1) Add "email.apps.local" to DNS
2) Configure CUPS used DNS (use "set network dns primary" command)
3) Use "email.apps.local" in presence gateway

Example #2

In a lab environment, you may configure the certificate so it identifies the OWA server with common name "192.168.1.100" (though it's not a common practice, it's technically legit).

In this scenario, you may use IP address 192.168.1.100 in presence gateway config.

Whenever you made changes to presence gateway, don't forget to restart Presence Engine service.

Exchange

You don't have to be an Exchange expert. But you should at least know the following:

OWA, WebDAV, Receive-As permission, IIS certificate, CAS/Mailbox roles, authentication methods, Power Shell. (did I say you don't have to be an expert? :) )

OWA

OWA - Outlook Web Access, a web interface to access Exchange emails, calendars, etc. HTTPS needs to be enabled on OWA for CUPS integration to work.

OWA is actually a very useful tool for troubleshooting. For example:

When you tried to access OWA via HTTPS, OWA server will present you a certificate. You may view that certificate from web browser. You should be able to tell what the CA cert is from the web browser.

In the screenshot below, you open a web page to OWA. If you double-click the "lock" icon at the bottom of IE, you'll see a certificate viewer window. From the "certification" tab, you'll see the CA cert is "sametime.apps.local". The identity cert is "owa.apps.local".



Now you know that:
1) You need to upload "sametime.apps.local" cert to CUPS as "Presence Engine - Trust".
2) You need to configure "owa.apps.local" as the "presence gateway" on CUPS > Presence > Presence Gateway.

WebDAV

WebDAV is a protocol (on top of HTTP) to retrieve email/calendar information. See http://en.wikipedia.org/wiki/WebDAV for more details.

CUPS uses WebDAV for calendar integration. Microsoft obsoletes WebDAV on Exchange 2010 and advocates its own protocol EWS (Exchange Web Service). By the time this blog was being written, CUPS doesn't support EWS yet. Which means, CUPS doesn't work with Exchange 2010. I'm not sure if you can have an E2007 CAS in E2010 as a "WebDAV" gateway. I'll test it later.

Though WebDAV sits on HTTP/HTTPS, it's different from OWA. Don't assume WebDAV is working just because OWA was working.

Up to Exchange 2007, WebDAV was enabled by default. You don't have to explicitly enable it. Some people reported that WebDAV was not installed/enabled on E2007 with Win2008. This is not true. The "WebDAV Publishing" in screenshot below has nothing to do with CUPS integration. It's OK to leave it uninstalled.


Since E2007, Microsoft change the OWA URL from /exchange to /owa, but WebDAV URL stays the same (/exchange). This caused more confusions in troubleshooting.

When you troubleshoot CUPS calendar integration, you'll see CUPS always request to URL /exchange, which is the WebDAV URL. This is right and expected.

If you tried to "test" it by typing "https://owa.acme.local/exchange", you'll be redirected to "https://owa.acme.local/owa". This is also right and expected. For more details, please see: http://msexchangeteam.com/archive/2007/02/07/434523.aspx

The point is: there's no easy way to tell if WebDAV is working. But it's easy to tell if WebDAV is NOT working.

If you typed "https://owa.acme.local/exchange" and got some error like "401 unauthorized" or "503 service unavailable", WebDAV is more than likely not working. Though regular OWA might be working fine at this point.

If you got "401 unauthorized", you may take a look at http://msexchangeteam.com/archive/2008/02/01/447989.aspx. Default settings should work. If it doesn't work, most likely your authentication was expecting username in a different format. e.g. "domain\user" instead of "user".

If you got "503 service unavailable" or "500 internal server error", please make sure "ISAPI Extensions" is installed on mailbox server. See screenshot below. This usually happens if mailbox server and CAS are on two different servers.

Certificate

OWA/WebDAV relies on IIS to server HTTP request. It's IIS that presents certificate to SSL client.

By default, when you installed IIS, it'll generate a self-signed certificate. Unfortunately, this self-signed certificate doesn't work with CUPS, because it doesn't contain CA bit in its extension. Below is an example of the certificate that HAS the CA bit:


If your CA cert does not have CA bit, CUPS won't trust it. This is per RFC 2459. Don't blame Cisco on that.

Solution? Get a certificate from a CA (either external or internal) or use "makecert.exe" to create a self-signed cert with CA bit (see my book for details).

Receive-As Permission

In order to see everybody's calendar, CUPS needs an account with this permission. So the FAQ is: what's the minimum permission required?

The minimum permission required is "Receive-As" on the end user's mailbox.

Cisco documents said you need "View Only Administrator". That's not true. All you need is "Receive-As" permission on the end user's mailbox. Not more, not less.

How do we assign "Receive-As" permission to the account? I would recommend use Exchange Management Shell (command line).

Let say, in Active Directory, the service account used by CUPS is called "cupsexch". The end user account is called "JDoe" (full name: John Doe). The following command give "cupsexch" Receive-As permission on John Doe's mailbox:

Add-ADPermission -Identity "John Doe" -User cupsexch -ExtendedRights Receive-As
Of course, you're not going to repeat this command for 1000 users. The better way to do it is to assign permission on a "container" that contains the end users' mailboxes.

In Exchange, the "container" for mailboxes is "Mailbox Database". The container for "Mailbox Databases" is "Storage Group". You may have multiple databases in a storage group. And you may have multiple storage groups in your Exchange environment.


You may assign permission at different levels:

End User Mailbox
Mailbox Database
Storage Group

Since "Storage Group" is the highest level of "container", we usually assign permissions on storage group level. Permissions would populate down to end user mailboxes. This "population" takes some time (from 30 minutes to hours).

The command to assign permission on "First Storage Group" is as below:

Add-ADPermission -Identity "First Storage Group" -User cupsexch -ExtendedRights Receive-As
What if you have multiple storage groups and want to automate the process? You may use the command below:

Get-StorageGroup | add-ADPermission –user cupsexch -ExtendedRights Receive-As
"Get-StorageGroup" command will get all storage group names from system and feed the names to "Add-ADPermission" command.

The command to verify permission is as below:

Get-MailboxPermission jdoe -user cupsexch | Format-Table -autosize
This command will display "cupsexch" permission on jdoe's mailbox. Anything after the pipe sign (|) is just for formatting purpose.

If you got nothing, that means "no permission". If you assigned the permission on higher containers, it might not have flowed down yet.

If the permission has been populated, you should see something like the screenshot below:


"AccessRights = {FullAccess}" means "cups7exch" account has permission to read "John Doe's" mailbox.
"Is Inhereited = True" means this permission was inherited from some higher level containers.

This is the only reliable way to test permissions.

Cisco documents described a test method which use OWA to test the URL "https://owa-address/exchange/some_user/calendar" with the service account. This method may or may not work depending on other permissions you configured for the service account. For example, if you are like me to give minimum permissions, Cisco test method would yield a "no permission" page as below. However, CUPS calendar works just fine with the "minimum permission".


Troubleshooting

If calendar integration doesn't work, "Presence Engine" (PE) logs are the only traces we need.

Ideally, we want the logs since PE started. Thus if you could, do the following:

1) Restart PE
2) Wait unti PE started and stabilized
3) Log into CUPC
4) Collect PE logs since the time of restart
5) Use WinGrep to search for keyword "owa". It'll give you all message regarding OWA transactions.

Following are some common problems:

"Certificate not trusted"

This could be caused by the following:

  • The name configured in "CUPS > Presence > Presence Gateway" does not match with the name in identity certificate. (e.g. IP address vs. FQDN)
  • CA certificate was not imported as "Presence Engine - Trust". If there are multiple CAs in the certificate chain, all of them need to be imported.
  • CA bit was not set (usually happens on IIS self-signed certificate)

"440 Login Timeout"

When FBA (Form-Based Authentication) was enabled, you'll see one "440 login timeout" in PE log for each transaction. This is normal. If you keep getting "440 login timeout", it usually indicates an authentication issue.

To reveal the nature of the problem, it's recommended to change "Form-Based Authentication" to "Standard Authentication" on "/Exchange" virtual directory on OWA. See screenshot below:


Changing this will not change the way or feel of regular OWA logon. It's just change the authentication method for WebDAV. (you'll need to reset IIS by using command "iisreset /noforce")

"401 Unauthorized"

This is usually caused by permission issue. See "Receive-As" permission configuration above for testing method.

Another testing method would be configuring an end user's credential on presence gateway. An end user's credential would definitely have permission on his own mailbox. If you log in to CUPC with that end user account, calendar integration should work. This is a good way to do "problem isolation".

To be continued...

21 comments:

  1. Thanks. Really nice job. I will definitely try this.

    ReplyDelete
  2. Hi Michael,

    first i have to thank you for your excellent book. It helped a lot installing and configuring presence, since i had absolutely no knowledge about it before. Now everything works...almost everything. After a lot of hassles with the certificates i got this working, but still can get no calendar integration working.

    03/17/2010 13:29:02.052 EPE|system.pe.pa.owa.backend 2063714 ERROR -->CalendarSubscription::initiateRecovery: myuser@mydomain.local SUBSCRIBE 3 Authentication failure on server; Could not authenticate to server: ignored NTLM challenge, ignoring empty Negotiate continuation, rejected Basic challenge

    Got still no idea why it isn´t working. I can logon to OWA without any problems using the cupsexch-user or the user seen in the error message, set the permissions as described, have no certificate problem...any ideas?

    ReplyDelete
  3. If you got authentication error, you may try the following:

    1) Go to CUPS > Presence > Gateway > Outlook Gateway.
    2) Change the service account (e.g. "cupsexch-user") to an end user account (e.g. "JohnDoe").
    3) Restart Presence Engine
    4) Log into CUPC with "JohnDoe" credential
    5) Test calendar integration

    If the feature works in this scenario, that means the problem was on the service account. Most likely permission has not been set up properly.

    If the feature doesn't work in this scenario, it's most likely your Exchange server expects a different format of logon ID. e.g. your Exchange server expects "domain\account" instead of "account".

    ReplyDelete
  4. Hi Michael,

    thanks for your response. I tried as described, but the problem still remains.

    As additional information...we don´t use LDAP authentication for CUCM and CUPS, only user synchronization. And we use other passwords for login to CUCM-/CUPS-userpages or CUPC. The cupsexch user is using his real password as used on our domain. For my test i had to use the real passowrd of "JohnDoe" too, since i can´t get Exchange-Access working (as expected) with our simplified passwords for the UC-environment.

    Could this perhaps be a reason for the _test_ not working?

    ReplyDelete
  5. The password you configured in CUPS > Presence > Gateway > Outlook Gateway should always be the AD password.

    If you used the end user account as integration account and it still failed, I guess you need to troubleshoot from Exchange side.

    Did you try different username formats as advised?

    ReplyDelete
  6. Hello Michael,

    I read your book carefully and it is really interesting.
    I meet today a problem with Calendar integration between CUPS 7 and Exchange 2007.
    Everything work fine but here are the IIS logs

    Fields: date time s-sitename s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status
    2010-02-08 14:13:36 W3SVC1 10.20.1.2 SUBSCRIBE /exchange/pwatrin@showroom.net/TypePEPresenceGatewayLocale.PEPRESENCEGATEWAYLOCALE_CALENDAR - 443 showroom.net\cupsexch 10.20.7.11 - 404 0 0
    2010-02-08 14:13:36 W3SVC1 10.20.1.2 SUBSCRIBE /exchange/cfriderich@showroom.net/TypePEPresenceGatewayLocale.PEPRESENCEGATEWAYLOCALE_CALENDAR - 443 showroom.net\cupsexch 10.20.7.11 - 404 0 0
    2010-02-08 14:13:36 W3SVC1 10.20.1.2 SUBSCRIBE /exchange/svos@showroom.net/TypePEPresenceGatewayLocale.PEPRESENCEGATEWAYLOCALE_CALENDAR - 443 showroom.net\cupsexch 10.20.7.11 - 404 0 0
    #Software: Microsoft Internet Information Services 6.0
    #Version: 1.0

    CUPS send a subscribe SUBSCRIBE /exchange/svos@showroom.net/TypePEPresenceGatewayLocale.PEPRESENCEGATEWAYLOCALE_

    And not SUBSCRIBE /exchange/svos@showroom.net/calendar

    I have installed the Locales but problem is still present.

    Do you have seen this problem before ?

    Thanks in advance,

    Stépahne

    ReplyDelete
  7. Hi,

    When I install the Certificate issued by the CAS on Exchange, OWA stops working. If I revert to the certificate generated by the local exchnage server OWA start working again.

    How do i resolve this

    ReplyDelete
  8. Not sure if I understand "Certificate issued by the CAS on Exchange, OWA stops working".

    CAS = Client Access Server = OWA.

    CAS does NOT issue certificate.

    Anyway, you should contact Microsoft (or your Exchange engineer) for technical support.

    ReplyDelete
  9. sorry, I mearnt to say my local CA. Was working into the early morning on this and could not get it to work.

    Basically when I replaced the Exchange server self generated certificate with the one issued by my local Certificate Authority, I could no longer access OWA. Just curious if anyone has had this issue. Thanks

    ReplyDelete
  10. Hi Michael,

    I succeffully uploades the certificate of the CA and not the one issued to exchange on the CUPS server. I have confirmed the the CUPSAdmin user have access to the user exchange mailbox. Also the exchange view only admin rights are granted to the CUPSAdmin user.

    changing my Microsoft Outlook 2010 status does not make the change in CUPC. I have enabled the user for calendering in CUPC.


    [PS] C:\Documents and Settings\Administrator.ENET>Get-ADPermission "First Storage Group" -user cupsadmin | Format-Table -autosize

    Identity User Deny Inherited Rights
    -------- ---- ---- --------- ------
    EXCHANGE\First Storage Group INET\CUPSAdmin False False Receive-As


    [PS] C:\Documents and Settings\Administrator.INET>Get-MailboxPermission jmooney -user cupsadmin | Format-Table -autosize

    Identity User AccessRights IsInherited Deny
    -------- ---- ------------ ----------- ----
    ENET.com/INET Employees/Managers/James Mooney INET\CUPSAdmin {FullAccess} True False

    I also tried login to my desk phone IPPM Services. Option 1 Todays Meeting gives the message

    The Calender server may be down. Please make sure the server is up and running.

    My presence gateway name is exchange.inet.com the same name used in the CN Field for the certificate request from exchange too the CA

    HELP!!! :-)

    ReplyDelete
  11. we're going to need Presence Engine logs to troubleshoot. Please upload logs to NetPro forum: https://supportforums.cisco.com/community/netpro/collaboration-voice-video/unified-comm-application

    ReplyDelete
  12. Thanks Michael,

    I have posted the logs on Net Forum with subject Calendar presence with CUPS and Exchange

    Also I ordered a copy of your book :-)

    Cheers

    ReplyDelete
  13. Michael,

    after reading your book and your blog we still are in trouble. We solved certificate and authentication issues, now we got:

    In IIS
    2010-09-15 14:22:19 172.27.1.2 SUBSCRIBE /exchange/john.doe@domain.com/calendar - 443 cupsexch@sg.local 192.168.70.12 - 404 0 0 15

    In Presence Engine Log
    16:22:20.049 |system.pe.pa.owa.backend 1958062 ERROR -->CalendarSubscription::initiateRecovery: john.doe@domain.com SUBSCRIBE 404 Ressource not found

    Any clues on this? We run Exchange 2007SP2 and CUPS8.0.2.10000-30

    Thanks

    ReplyDelete
  14. "404 resource not found" means Exchange couldn't find the mailbox for "john.doe@domain.com". What is "domain.com" and what is "sg.local"?

    ReplyDelete
  15. Hi Michael,

    sg.local is our internal domain, the same as domain.com, i just forgot to anonymize the first. ;)
    It reads the same in the log.

    And the mailbox exists, that´s whats kind of funny, we can log on via OWA on this mailbox.
    It also has the needed permissions.

    ReplyDelete
  16. Hello Michael, our problem is solved:

    Cisco bug - CSCsz12455

    Could be interesting if someone else is experiencing the same problems.

    Best regards,
    Andreas

    ReplyDelete
  17. We had the same error. Turned out to be localized calendar names. Presence server checked for /kalender and it turned out to be that the user was using /calendar.
    Outlook.exe /resetfoldernames helped out.

    ReplyDelete
  18. hi Micheal,

    I've read Exchange/CUPS integration section a few time already and unfortunately, I can not get it to work. The certificate verification is failing. My customer have Verisign as the CA. The CA is issued tothe OWA FQDN. This FQDN points external public ip address on a firewall. There is no issue with the FQDN, user can access webmail over the internet.

    My problem is, since the FQDN for the OWA is an external IP address and it does not response to PING. I can not simply do a ping test on the CUPS to confirm the network connection is working.

    Now, if the user is accessing webmail from the internal network, they have a different internal URL, which points directly to the exchange server internal IP address.

    I know for sure CUPS can reach the internal ip address of the Exchange. So I would like to know if I can still use makecert.exe on the OWA server to generate a self signed certificate, even though the customer already have an external CA (Verisign) ?

    One interesting thing I found is, every time the certification failed on CUPS, it delete the exchange certificate, so I would have to upload it again to test. Is this normal ?

    ReplyDelete
  19. Hi Sir,
    i'm actually working on a presence failure between the Presence Server v8.0.4 and the Exchange v 2003. it seems that the presence server lot communication with the exchange server and i will like to know what is the default timeout timer cups server waits for an answer from the exchange server and if this timer can be modified?

    here is the error log
    : 1046: server1.domain1.com: Feb 14 2011 22:08:09.234 UTC : %UC_-3-PECalendarConnectionLoss: %[UNKNOWN_PARAMNAME:PEAlarmMessage=exchangebalancefarm:443 Connection timed out; Could not read status line: connection timed out][ClusterID=StandAloneCluster40350][NodeID=server1]: Indicates that the Cisco UP Presence Engine service cannot connect to the Calendar Server

    thanks for you answer

    ReplyDelete
  20. Hello,
    I´m trying to connect CUP Server8.5 with Exchange 2010. Do you have any information for me? Gateway is connected, Mailbox read permission is O.K., but the status for calendar integration is disabled. Any ideas?
    Thanks Francisco

    ReplyDelete