Establishing Credibility–Yay or Nay?

A discussion among many good friends that I respect started out with one friend saying how much he hated presentations where the speaker spends time on the slide introducing themselves. I agree.

There is a reason, and a bit of a story that goes with my view.

About 20 years ago, I attended a sales seminar that was scheduled for four days. I had nothing on the books with a customer, so I signed up and hoped that I would be able to get some good information that I could apply to my work as a consultant.

Day 1 at 9am – In walked this short Iranian with a turban. He looked like he was about 60 or so. He was wearing khakis and a very basic white button-up shirt. No flash. He was the speaker/presenter. He came in, said hello, gave his name, and dived right into the subject. I remember, very clearly, that I was kind of turned off by his appearance, and I remember that I was thinking about ditching the rest of the week. All of these thoughts went through my head about how I was wasting my money. I was starting to plan my exit during lunch.

Day 1 at 9:15am (ok, it might have been 10 minutes in) – While, I was a little shocked with his appearance, he immediately struck me as a true professional. He walked us through his plans for the next four days, and threw out some scenarios that we would cover in each section of the seminar and that he would not just cover them in depth, but he would also explain how different methods would work or not work and why.

Day 1 at lunch – After completing several exercises and discussing several scenarios. I didn’t care if he was an alien from another planet. The guy was good. It was clear that he was the real deal. Actually, it was clear within the first few minutes, that it was going to be worth my time.

Day 4 at 9am – He walked in wearing a beautiful suit and was not wearing his turban. His long flowing grey hair was amazing and contrasted perfectly with his suit. This is what I was expecting when he first came into the room on Day 1. He was rocking it!

Day 4 at 3pm – After four days of incredible content and great information, he started to sum up the week, and at that point, he threw his resume up on the projector. Undergrad at Yale, MBA at Harvard, years of work experience with the biggest of the Fortune 500 companies at the highest levels. He resume was amazing and impeccable.

I asked him why he didn’t share his resume to start on Day 1 to establish his credibility. His response said it all (I am paraphrasing just a bit as I remember it very clearly), “You do not need to establish your credibility if you know your subject and you clearly care about helping those in front of you. Your value is in what you do to help those in front of you, not what you tell others about your past. You demonstrate your value through your words and actions, not through a piece of paper that talks about your past.”

Advertisement
Posted in Uncategorized | Leave a comment

OOF Messages Sent Multiple Times

 

First off, it is OOF, not OOO. It is Out of Facility, not Out of Office.

A few days ago, I received a request asking me why people were getting the OOF from a user that was out, even though they had received it already. In other words, you are only supposed to get the OOF response once during the time frame, so why were people getting replies every single time.

At first, I thought somebody just set up a mailbox rule, rather than using the Automatic Replies option. After checking the mailbox, I saw that the Automatic Replies was set correctly. After a bit, I remembered some peers talking about this issue, and I found the article. Thank you Bing.

https://support.microsoft.com/en-us/help/3106609/out-of-office-oof-messages-are-sent-multiple-times-to-recipients

Summary, the history of responses is tracked, and there is a limit of 10,000. I still can’t  believe that this limit was actually exceeded. In this case, it was a user that gets hundreds of emails a day, and they were out on medical leave for almost three months.

I am shocked that it happened. I am even more shocked that I remembered that this was an issue and found the article.

Posted in Uncategorized | Leave a comment

Get-CsBackupServiceStatus ErrorState

Before performing an Invoke-CsPoolFailover, it is always a good idea to make sure that the Backup Service is running properly. You should run the cmdlet from both sides to verify that you have

 

Get-CsBackupServiceStatus -poolfqdn pool01.infrastructurehelp.com

 

RunspaceId          : be5ae52b-3f48-4d15-8d36-842d4f5cacf6

ActiveMachineFqdn   : fe01.infrastructurehelp.com

OverallExportStatus : SteadyState

OverallImportStatus : ErrorState

BackupModules       : {UserServices.PresenceFocus:[SteadyState,ErrorState],

                      ConfServices.DataConf:[SteadyState,NormalState],

                      CentralMgmt.CMSMaster:[NotInitialized,NormalState]}

 

 

Get-CsBackupServiceStatus -poolfqdn pool02.infrastructurehelp.com

 

RunspaceId          : be5ae52b-3f48-4d15-8d36-842d4f5cacf6

ActiveMachineFqdn   : fe4.infrastructurehelp.com

OverallExportStatus : SteadyState

OverallImportStatus : NormalState

BackupModules       : {UserServices.PresenceFocus:[FinalState,NormalState],

                      ConfServices.DataConf:[SteadyState,NormalState],

                      CentralMgmt.CMSMaster:[SteadyState,NotInitialized]}

 

Above, you can see that the import on pool01 failed. An issue can occur when moving a large number of users between pools where the databases don’t properly synch up during the moves. In such a case, there might be a user account, or more, that appears in both databases.

 

Microsoft support can provide a copy of the PoolConflictCorrector tool, which reads through the databases and identifies any user accounts that appear on both sides of the environment. If there are any duplicates found, the tool will prompt you to correct each one. This will fix most issues with Backup Service failures.

 

.\PoolConflictCorrector.exe -pool1fqdn pool01.infrastructurehelp.com -pool2fqdn pool02.infrastructurehelp.com -logfile PCC.log

 

Checking data in backend database in both pools…

 

Below entities have pool ownership conflict.

      User mother.goose@infrastructurehelp.com

      User jack.b.nimble@infrastructurehelp.com

 

The tool will find the right ownership for each conflicted entity and ask for your confirmation before correcting the conflict in backend database. Are you ready to continue? Type YES to continue. Type any other to abort: YES

 

User mother.goose@infrastructurehelp.com is owned by pool pool01.infrastructurehelp.com. Please double check and confirm the pool ownership before correcting the conflict in backend database. Type YES to continue. Type any other to skip this entity: YES

 

Conflict for User mother.goose@infrastructurehelp.com have been successfully corrected.

 

User jack.b.nimble@infrastructurehelp.com is owned by pool pool13bdc.schwab.com. Please double check and confirm the pool ownership before correcting the conflict in backend database. Type YES to continue. Type any other to skip this entity: YES

 

Conflict for User jack.b.nimble@infrastructurehelp.com have been successfully corrected.

 

While the issue is fairly rare, the tool is very easy to use and seems to resolve the issue. By the way, the “have” statement in the results is actually the grammar as provided by Microsoft.

Posted in Uncategorized | Leave a comment

Invoke-CsPoolFailover Warning

If you read all of the documentation, the Invoke-CsPoolFailover cmdlet sounds fantastic and pretty straight forward.  In reality, there are challenges. I haven’t tested a failover for a long time in my lab. So, I figured “Why not?”

Invoke-CsPoolFailOver -poolfqdn pool01.infrastructurehelp.com

I went through the confirmations, and got this nice warning/error.

WARNING: Invoke-CsPoolFailOver encountered errors. Consult the log file for a detailed analysis, and ensure all errors (1) and warnings (0) are addressed before continuing.

WARNING: Detailed results can be found at “C:\Users\russ.kaufmann\AppData\Local\Temp\11\Invoke-CsPoolFailOver-54fa4569-bb52-4c3b-8719-6ec7e4e01c2f.html”.

Reading further, got me this pretty clear message.

Invoke-CsPoolFailOver : This Front-end pool “pool01.infrastructurehelp.com” is specified in topology as the next hop for the Edge server. Failing over this pool may cause External access/Federation/Split-domain/XMPP features to stop working. Please use Topology Builder to change the Edge internal next hop setting to point to a different Front-end pool,  before you proceed.

The fix is pretty simple. You need to point the next hop for the Edge server to point to the other pool However, it isn’t clear that you can only do it via PowerShell. The cmdlet is pretty simple, though.

Set-CsEdgeServer -Identity EdgeServer:edge01.infrastructurehelp.com -Registrar Rgistrar:pool02.infrastructurehelp.com

Running the cmdlet will get everything sorted out so you can run the Invoke-CsPoolFailover. However, it will generate a nice warning to tell you that there is a problem if your Front-End pools and Edge pools are in different sites. Of course, to me, that is the best reason to have multiple sites and pools.

WARNING: ServiceId “EdgeServer:edge01.infrastructurehelp.com” depends on “Registrar:pool02.infrastructurehelp.com” from different site for dependency ID “Registrar.Default”.

The end result: I wrote this blog post so that I can use it in case I actually have to do a disaster recovery and move everything to another pool.

Posted in Uncategorized | Leave a comment

Troubleshooting Lync Server 2013 Federation

The other day, I noticed that one of my contacts in Lync was showing Presence unknown. Fed1

Well, this does happen as people change jobs and go to different companies. I wondered about it, so I asked a common friend if he had heard anything. His response was that they were still there, and something must be wrong with my Federation configuration.

I was pretty sure that nothing was wrong with my Federation configuration since I could see others from other companies and was able to IM them and have conferences with them.

Basic Troubleshooting

I always start with the very basics. Has anything changed recently? What do the Event logs show?

What changed recently is that I migrated to a new Access Edge server. Initially, I discounted that as Federation was working for every other company,  so I didn’t think it was the server configuration.

Next, I opened Event Viewer and checked my Lync Server logs. I admit, I don’t always go to the Event logs early on in my troubleshooting. I have no valid reason, I just don’t do it. In this case, I did. I am also glad that I did, because this is what I found:

TLS outgoing connection failures

Over the past 12 minutes, Lync Server has experienced TLS outgoing connection failures 31 time(s). The error code of the last failure is 0x80090325(SEC_E_UNTRUSTED_ROOT) while trying to connect to the server “edge.CompanyName.com” at address [10.100.100.200:5061], and the display name in the peer certificate is “Unavailable”.

Cause: Most often a problem with the peer certificate or perhaps the host name (DNS) record used to reach the peer server. Target principal name is incorrect means that the peer certificate does not contain the name that the local server used to connect. Certificate root not trusted error means that the peer certificate was issued by a remote CA that is not trusted by the local machine.

Resolution:

Check that the address and port matches the FQDN used to connect, and that the peer certificate contains this FQDN somewhere in its subject or SAN fields. If the FQDN refers to a DNS load balanced pool then check that all addresses returned by DNS refer to a server in the same pool. For untrusted root errors, ensure that the remote CA certificate chain is installed locally. If you have already installed the remote CA certificate chain, then try rebooting the local machine.

When I deployed my new Access Edge server, I made sure that I added all of the major Certification Authorities to the Trusted Root Certification Authorities container. I realized that I must have missed at least one.

Digicert to the Rescue

I have always been a fan of Digicert, mostly because of how they support the MVP community, but also because they provide several great tools.

First, I opened a command prompt and ran nslookup.exe. From there, I set the type to SRV, and then put in _sipfederationtls._tcp.companyname.com, and it provided me the name of the Access Edge server on the other side, edge.companyname.com.

Fed3Next, I went to the Digicert site: https://www.digicert.com/help/ and put in the edge.companyname.com record into the tool. I scrolled down, and found the Certificate Authority that they were using for their certs on edge.companyname.com.

Yes, I did not have this particular Go Daddy Certificate Authority in my Trusted Root Certification Authorities list.

The last step was to go to https://certs.godaddy.com/repository and get the CA Cert.

Ta da! Problem fixed.

Posted in Uncategorized | Leave a comment

Lync Server 2013–No Users Homed on Front-End

It is funny how you find odd behaviors and then try to figure out what is wrong or why something doesn’t behave like it should.

This situation all started with one of Pat Richard’s famous PowerShell one-liners. Pat, for those that don’t know him, is a wealth of information, and he has helped me out on many occasions. In this case, I was perusing his one-liners and saw one that I liked. However, I didn’t like the output and wanted to use it in a bit of a different way than his example. What did I do? Well, I took a perfectly good one-liner and made it into a mutli-line PowerShell script.

CLS

$CsServers = (Get-CsComputer | where {$_.identity -ilike “fe*” -and $_.pool -ilike “pool13*”}).identity

Write-Host ” “

Write-Host “Active Lync Server 2013 Connections”

Write-Host “———————————–“

ForEach ($cs in $CsServers) {

                # Write-Host Checking Counters on $cs

                $EndPoints = (Get-Counter “\LS:USrv – Endpoint Cache\USrv – Active Registered Endpoints” -ComputerName $cs | select -expand CounterSamples).CookedValue

                $RegUsers = (Get-Counter “\LS:USrv – Endpoint Cache\USrv – Active Registered Users” -ComputerName $cs | select -expand CounterSamples).CookedValue

                Write-Host $cs `t “EndPoints: $EndPoints” `t “Users: $RegUsers”

}

What was interesting is what I found. My pilot users were not spreading across my pool like I thought they would. While I guess that the numbers might even out more as I add users, the “0” really bothered me.

fe1.infrastructurehelp.com          EndPoints: 17   Users: 9

fe2.infrastructurehelp.com          EndPoints: 4    Users: 4

fe3.infrastructurehelp.com          EndPoints: 19   Users: 15

fe4.infrastructurehelp.com          EndPoints: 5    Users: 5

fe5.infrastructurehelp.com          EndPoints: 0    Users: 0

At first, I was thinking, yep, just a statistical anomaly. I talked to one of my colleague, Pete Holmes, and he thought it was strange, too, and he started digging into it. Pete tested the impact of changing his SIP URI to change the impact of the algorithm as described here: http://blogs.technet.com/b/nexthop/archive/2011/05/25/dns-load-balancing-in-lync-server-2010.aspx The Blog entry says that the “registrar assignment is calculated by a hash value of the user’s SIP URI.” The idea was that changing the SIP URI would result in him getting placed on a different Front-End server. No matter how he changed his SIP URI, it didn’t put him on fe5. Setting the client connection settings to force the client to connect to fe5 resulted in it being properly redirected to the proper front-end server, so it appeared that the server was working correctly. Nothing was found in Event Viewer pointing to any issue, either. Basically, the server looked like it was broken and looked like it was fine all at the same time. Schrödinger’s Front-End Server?

We talked about it for a bit, and I thought it might be that the routing groups are set up so that FE5 is a secondary/tertiary for all of the routing groups. That didn’t sound like it was possible, again, it sounded like a statistical anomaly. No matter how I looked at it, it just didn’t seem right. Pete checked the Fabric, and there it was. Highlighted below, it was clear that none of the routing groups were primary for fe5.

PS > Get-CsPoolFabricState -poolfqdn pool13.infrastructurehelp.com

Replica Instances for MCUFactory Service

    Address: fe1.infrastructurehelp.com – Primary: 1 Secondary: 3

    Address: fe2.infrastructurehelp.com – Primary: 1 Secondary: 3

    Address: fe3.infrastructurehelp.com – Primary: 1 Secondary: 3

    Address: fe4.infrastructurehelp.com – Primary: 1 Secondary: 3

    Address: fe5.infrastructurehelp.com – Primary: 2 Secondary: 0

Replica Instances for ConferenceDirectory Service

    Address: fe1.infrastructurehelp.com – Primary: 1 Secondary: 0

    Address: fe2.infrastructurehelp.com – Primary: 0 Secondary: 1

    Address: fe3.infrastructurehelp.com – Primary: 1 Secondary: 0

    Address: fe4.infrastructurehelp.com – Primary: 0 Secondary: 1

    Address: fe5.infrastructurehelp.com – Primary: 0 Secondary: 2

Replica Instances for Routing Service

    Address: fe1.infrastructurehelp.com – Primary: 9 Secondary: 17

        Local Groups: Primary: 2 Secondary: 3

        Remote Groups: Primary: 7 Secondary: 14

    Address: fe2.infrastructurehelp.com – Primary: 8 Secondary: 17

        Local Groups: Primary: 1 Secondary: 3

        Remote Groups: Primary: 7 Secondary: 14

    Address: fe3.infrastructurehelp.com – Primary: 10 Secondary: 16

        Local Groups: Primary: 3 Secondary: 2

        Remote Groups: Primary: 7 Secondary: 14

    Address: fe4.infrastructurehelp.com – Primary: 8 Secondary: 16

        Local Groups: Primary: 1 Secondary: 2

        Remote Groups: Primary: 7 Secondary: 14

    Address: fe5.infrastructurehelp.com – Primary: 7 Secondary: 18

        Local Groups: Primary: 0 Secondary: 4

        Remote Groups: Primary: 7 Secondary: 14

Replica Instances for LYSS Service

    Address: fe1.infrastructurehelp.com – Primary: 9 Secondary: 17

    Address: fe2.infrastructurehelp.com – Primary: 8 Secondary: 17

    Address: fe3.infrastructurehelp.com – Primary: 10 Secondary: 16

    Address: fe4.infrastructurehelp.com – Primary: 8 Secondary: 16

    Address: fe5.infrastructurehelp.com – Primary: 7 Secondary: 18

Global Service Count Summary:

Fqdn: fe1.infrastructurehelp.com – Primary: 20 Secondary: 37

Fqdn: fe2.infrastructurehelp.com – Primary: 17 Secondary: 38

Fqdn: fe3.infrastructurehelp.com – Primary: 22 Secondary: 35

Fqdn: fe4.infrastructurehelp.com – Primary: 17 Secondary: 36

Fqdn: fe5.infrastructurehelp.com – Primary: 16 Secondary: 38

How do you fix it? Don’t worry about it, it will take care of itself over time.

Posted in Uncategorized | Leave a comment

Issue with Meet Now and Invite More People–RCC and Lync Server 2013

Disclaimer: Yes, I know that Remote Call Control (RCC) is being deprecated. Yes, I know Enterprise Voice is WAY better.

A “feature” that was found is that users that are RCC enabled are not able to add other users to meetings that are started by using Meet Now. When meetings are scheduled and invitations are sent via email, users can join by clicking on the link in the email, and all is well. OK, not so much, so let me rephrase, all is well UNTIL another person is invited by using the Invite More People option.

What happens in both cases is that you, as a person in the meeting,  will see the users pop into the meeting for a couple of seconds, then they will disappear. From the user’s side, they start to join the meeting, then they get an error where audio fails to connect (if the meeting is not using Lync audio, i.e. a call bridge or just a shared session) and they get a message at the bottom of their Lync client to retry the connection for audio. If they select the option to rejoin using Lync audio, it will allow them to connect to the meeting.

This strange behavior does not appear if the users are not RCC enabled. If the user is configured for PC to PC audio only, or if they are Enterprise Voice enabled, there are no issues.

This is what appears to happen:

  1. The user that attempts to join the meeting sends a SUBSCRIBE request
  2. The user receives a 200 OK  with the conference information
  3. The user sends an INFO request containing the endpoint URI and the user’s LineURI using a “Dialed-Out” join method
  4. The user is not enabled for dial-out since they are not Enterprise Voice enabled, and they receive a 404 Not Found

So, basically, RCC users joining a conference that does not allow dial out for non-Enterprise Voice users receive a failure to join audio. This is by design. It is not a bug. This only happens when the user is directly invited into a meeting (via Meet Now and Invite Other People), and it does not happen if the user connects to the conference through the Meet Simple URL. After the failure, the user receives the option to rejoin or retry. If the user selects Lync Call, they will then connect. 

Summary: Yes, you should not use RCC, and you should migrate to Enterprise Voice, if it is possible.

Posted in Uncategorized | Leave a comment

Missed Call Notification Failure

Who knew that users would actually get upset if they don’t get their Missed Call emails? Who knew that they would also get upset if they received hundreds of notifications in their email that they missed all of those calls while out on maternity/paternity leave? Who knew that users would get upset no matter what?

Oh wait, all of us admins knew. Smile

Not much has changed in the last several years regarding how the Missed Call Notification works. Jens Trier Rasmussen covers the process very nicely, and it is still pretty accurate, today.

Recently, I have had a couple of users complain that they are not getting their notification emails. The answer, in these cases, has been simple. Would you guess it is because they have too many folders in Outlook? Yes, it is the same issue as with Conversation History not being saved after it has worked for a long time. Exchange Web Services has an issue when there are over 1,000 folders in the mailbox. Once the mailbox is cleaned up, the Missed Call Notification starts working again.

Of course, the best solution might just be to turn it off for everyone by using a Client Policy: Set-CsClientPolicy “PolicyName” -EnableCallLogAutoArchiving $False

If users want to disable the notifications, they can do it in the Lync/Skype for Business client application: In Lync/Skype for Business, click the Gear in the upper tight, then click on Tools, Options, then click on the Personal tab and disable the checkbox for Save call logs in my email Conversation History folder.

Posted in Uncategorized | Leave a comment

Address Book Regeneration–Different in Lync 2013

As often happens, I miss some interesting changes between versions of OCS/Lync/Skype for Business.

This time, I missed the following changes:

RtcAb and RtcAb1 – In Lync Server 201o, the Address Book uses two databases, rtcab and rtcab1. The two databases are used so that one database can respond to queries while the other one is updated at 1:30am. This way, there is always an active version available, and then when the other one is finished updating, it becomes the active one.

In Lync Server 2013, the rtcab responds to queries while it is updating and the second version is not required.

Manual Address Book Update – In Lync Server 2010, the manual process to update the address book requires two different PowerShell cmdlets. The process is to run Update-CsUserDatabase, then after the update completes, you run Update-CsAddressBook to sync the content.

In Lync Server 2013, all you have to do is run the Udpate-CsAddressBook. The cmdlet triggers the user replicator process as part of the process.

I guess I need to take more time away from actually working and doing more research. Smile

Posted in Uncategorized | Leave a comment

Lync 2013 Client and Remote Call Control – CUCM

Yes, I know. Remote Call Control (RCC) is dead in the world of Lync. Well, it’s not dead yet, but I am sure it will get fully clubbed over the head soon.

This issue reared its ugly head the other day when upgrading Lync 2010 clients to Lync image2013 clients. Every time you would start Lync 2013, it would generate this warning pop-up:

Cannot apply your new call forwarding settings. To turn off call forwarding, click OK. To ignore this error, click cancel.

If you click OK, it goes away until the next time you sign into Lync 2013. If you click Cancel, it goes away until the next time you sign into Lync 2013. Basically, nothing gets rid of it. This does not happen if you are using Lync 2010, though.

The reason? Well, Cisco… Yep, it isn’t a Lync 2013 issue, even though it appears to be it’s fault. Lync 2010 doesn’t experience because it doesn’t care what the Cisco Unified Call Manager tells it.

Basically, the issue is that CUCM tells the Lync 2013 client that call forwarding is set, even though it isn’t. See the Cisco Bug report for more info. The good news is that Cisco is going to fix it. Hopefully, it won’t be too long.

Posted in Uncategorized | Leave a comment