Tuesday, January 17, 2017

Skype for Business Rate My Call Viewer Tool

Rate My Call is a feature in Skype for Business that provides enterprises a way of acquiring feedback from their end-users via a special dialog window that pops up after a specified number of calls. The Rate My Call dialog window offers a combination of star rating system, predefined feedback checkboxes for audio and video calls, as well as the option for custom text based feedback. This gives the administrator a method for adding real user feedback to the existing Quality of Experience statistics that have been available since Lync was first introduced.



Rate My Call Prerequisites


In order to use the Rate My Call feature you will need the following pre-requisites:
  • You must have Skype for Business Server installed;
  • Users need to have a client version 15.0.4711.1002 or later and using the Skype for Business UI;
  • The RateMyCallDisplayPercentage in Client Policy must be set to a value larger than 0;
  • Users must be homed on a Skype for Business Server front end pool; and
  • The Skype for Business environment must have a monitoring database deployed.

Rate My Call Settings


The Rate My Call feature has two settings within Client Policy: Display Percentage and Allow Custom User Feedback. The Display Percentage is the percentage of calls that the user will be asked to provide feedback on. The percentage number is very important because if you set this value too high it can result in user survey fatigue whereby users stop providing relevant feedback due to being asked too often. The Allow Custom feedback setting is used to give users the ability to offer specific text based feedback in addition to their star rating and standard checkbox responses. The custom feedback can be great for drilling deeper into what the actual issue may be that the user is experiencing, rather than trying to interpret checkboxes responses that may not exactly match the user's experience.

There is no action required to enable the base feature, however custom feedback will need to be enabled separately if it is desired. The Rate My Call feature is automatically enabled in the Client Policy with the following defaults:
  • Rate My Call Display Percentage – 10%
  • Rate My Call Allow Custom User Feedback – disabled
If you have deployed Skype for Business and not changed the defaults, you may already have some data saved within the Monitoring database that you can start analysing.

The following Windows PowerShell cmdlet is an example of enabling custom end user feedback and changing the interval from 10% to 80%.

Configuring Rate My Call:
Set-CSClientPolicy -Identity <PolicyIdentity> -RateMyCallDisplayPercentage 80 - RateMyCallAllowCustomUserFeedback $true

The feature can be completely turned off by setting the RateMyCallDisplayPercentage to 0%.


Accessing Rate My Call Data


When the Rate My Call feature was introduced in Skype for Business Server, there was no interface added to the in the Skype for Business Monitoring Reports interface (which is still true to this day). There was, however, access added in the Call Quality Dashboard product which often doesn’t get deployed due to the overhead of additional SQL server(s) infrustructure. Technet does offer a couple of SQL query examples for getting the basic data out of the system. However, this is not particularly user friendly, so I thought I might make a simple Powershell tool for pulling out data and visualizing it so you can start compiling the user feedback that you may already have stored in your Monitoring database.


Rate My Call Viewer Tool




Features:
  • Select your required start and end date, rating filter (above or below the selected rating), SIP URI filter, Reason filter, and Voice and/or Video to be listed up. Note: the filters use regex, so for example you could use it to filter for multiple reasons using the OR Operator like this “echo|backgroundnoise”.
  • Export all events into CSV format.
  • Create graphs (Stars Bar Graph, Stars Pie Graph, Reason Pie Graph, Reason Bar Graph, Type Pie Graph, Stars Stacked Bar Graph, Trend Over Time Line) of your Call Rating data.

Version 1.01 Update:
  • Added the ability to select individual monitoring servers from a drop down box. This was added for large environments that have multiple monitoring databases and only want to retrieve statistics from one at a time. By default, all monitoring database will be queried.
  • Added a check for the database version. The tool only works on Skype for Business databases so the check makes sure the database is at least Version 7 (ie. Skype for Business level).
Version 1.02 Update (20/04/2017)
  • Added date/time localisation checkbox. By default the monitoring server records time is in GMT. This update adds a checkbox to localise all the date/time values to be in the timezone of the server you are running it on (instead of GMT). This changes the date pickers as well as the date displayed in the list and graphs.
  • Added the ability to zoom in on the Trend Over Time chart. You do this by clicking and dragging the mouse on the area of the graph you want to zoom to, scroll bars will appear so you can scroll the zoomed in view.
Version 1.03 Update (22/04/2017)
  • Fixed an issue with the SQL query used for Video / Audio. The query now gets all records.
  • Fixed issue with data grid view scroll bar refresh.
  • Fixed a sorting issue with the Stacked Bar and Trend Over Time Graphs that would cause an issue with the output.
  • More accurate graphs! When both video and voice are selected the rating data gets listed twice for each call because video calls contain both voice and video ratings. So in previous versions the star ratings were counted as separate calls which artificially inflated the star rating value given. In this version the double counting of this data has been removed from star rating graphs, with the voice and video star rating given by each user being combined. 
Version 1.04 (15/5/2017) – C2R Update
  • Now Supports Skype for Business C2R 2016 client Rate My Call issue items. The C2R 2016 client has an entirely new set of rate my call feedback, so the tool has been updated to include these.
  • Re-worked the graphs again to handle new data
  • Voice and Video calls don't get listed twice in this version (as it did in the previous version), graph processing was updated from previous version to handle this.
  • Get Records processing speed was increased by limiting records by date range in SQL query.
1.05 Update (16/3/2018)
  • Total Rows Counter added at the bottom
  • "Top 10 One Star Users" graph added. This can be used so you can follow up with these users about their bad experiences.
  • "Top 10 Zero Star Users (Lync 2013 Client)" graph added. This can be used to follow up on Lync 2013 client users that are not responding the Rate My Call dialog.


The C2R release of Skype for Business 2016 client brought a new look to the UI and in the process also introduced new Rate My Call dialog with all new Issues selections. Whilst the Skype for Business on premises server doesn’t have definitions in the database for these values it does save the new TokenIDs when users rate calls. Which means I was able to add these new values to the Rate My Call Viewer Tool.

Old and New Rate My Call dialogs

I have had to abbreviate these sentences into values that can be easily displayed in the tool. I believe the abbreviated names are fairly self-explanatory, however, here’s a list of the names I’ve used:

Type
Issue Sentence
Abbreviated Name
Relation to old values
Audio
I could no hear any sound
NoSpeechNearSide
New
Audio
The other side could not hear any sound
NoSpeechFarSide
New
Audio
I heard echo in the call
Echo
Existed in original
Audio
I heard noise on the call
IHeardNoise
New
Audio
Volume was low
VolumeLow
New
Audio
Call ended unexpectedly
VoiceCallCutOff
New
Audio
Speech was not natural or sounded distorted
DistortedSpeech
New
Audio
We kept interrupting each other
TalkingOverEachOther
New
Video
I could not see any video
NoVideoNearSide
New
Video
The other side could not see my video
NoVideoFarSide
New
Video
Image quality was poor
PoorQualityVideo
New
Video
Video Kept Freezing
FrozenVideo
Existed in original
Video
Video stopped unexpectedly
VideoCallCutOff
New
Video
The other side was too dark
DarkVideo
Existed in original
Video
Video was ahead or behind audio
VideoAudioOutOfSync
New



Prerequisites:
  •  This tool should be run on a machine that has the Skype for Business powershell module installed. This is required because the "Get-CSService" command is used to discover the location of the Montoring Database.
  • The user running the tool needs to have sufficient rights to run select queries on the "QoEMetrics" database and SELECT access on the following tables: Session, AudioStream, CallQualityFeedback, CallQualityFeedbackToken, CallQualityFeedbackTokenDef, User, MediaLine

Download from Technet Gallery:




Built-in Graphs


The Stars Bar Graph is the simpliest way to quickly see the how the users are rating calls on the system. In general you can ignore 0 star ratings because they these will be non-rated calls.



The Reason Bar Graph allows you to easily see trends in the number of each type of issue reported by users. This can be useful for picking out specific types of problems in your network.


  
The Reason Pie graph give you a quick view of which types of issues are most prevalent within your environment.


  
The Starts Pie Graph give you an idea of the how pleased your users are overall with the quality of calls within the environment.



The Stars Stacked Bar Graph allows you to differentiate between the ratings of Video and Voice calls separately from each other. This will allow you to understand if the either of the modality types is having more issues than the other. This will allow you to make choices about what to focus your future troubleshooting on.



The Type Pie Chart gives you an extremely simple depiction whether users are having more issues with Voice or Video calls within the environment.



The Trend Over Time Line is useful for tracking over time how the call quality has been changing within the environment. Note: if you are graphing over a long period of time it can be useful to maximize the graph window to see more details in the graph.


The “Top 10 One Star Responders” graph will show you the users that have responded with 1 star the most often. This can be useful to help you find users that are having issues with the system so you can contact them directly to follow up one-to-one to address their problems.



The “Top 10 Zero Star Users (Lync 2013 Client)” shows the top 10 users that have not being responding to the Rate My Call dialog box. Some organisations set their feedback dialog percentage in the system to be 100% in order to get feedback from users during trials, etc. So this can be useful to find users that aren’t responding with feedback. Note, this only applies to the Lync 2013 client using the Skype for Business UI. The Skype for Business 2016 client does not log 0 stars anymore when user don't respond. In fact, when using the 2016 client the user must select a star value between 1-5 in order to log any feedback in the database.



The Wrap Up


Well there it is! Now listening to your users is as simple by turning on the Rate My Call feature (which by default is already on!) and using this tool to extract and graph your data. In addition to the standard QoE statistics that the system offers, the Call Rating System built into Skype for Business can be an invaluable tool to understand your network quality. Enjoy!



14 comments:

  1. Good stuff, I've been looking for something like this. Thanks!

    ReplyDelete
  2. Can we get an option to chose or multi-select the monitoring databases that this tool queries on please? We have several monitoring databases and would like to chose the monitoring DB before the tool can start its query.

    ReplyDelete
    Replies
    1. Hi Yashie, I have added the ability to select individual monitoring servers in version 1.01.

      Delete
  3. Hi,

    Great Tool but i often get double entries for the same Call. You have one example in your Screenshot (2016/11/14 22:22:06, two entries for the same call).

    ReplyDelete
    Replies
    1. Hi Martin, these multiple entries at the same time (fer example in the screenshot) are different people reporting on the same call. If you look in the Caller column you will see that it's a different user reporting on the same call. When I was testing I had the feature configured to show for 100% of calls, so all users were reporting on each call. So this is expected.

      Delete
  4. Hi James,
    Trying to use this tool and am not surfacing any recent results. I know the data is there, as can be found in the dbo.CallQualityFeedback table in the QoEMetrics DB. For some reason, the tool is pulling the same data that is pulled using the query on the technet article that shows you how to extract it. I also can see the feedback data in EventZero's UCC. I rather use this tool to be honest, as it's easier to read than clicking thru a bunch of calls in UCC. Any advice? Thanks!

    ReplyDelete
    Replies
    1. Howdy,
      So the monitoring database dates are all saved in GMT time. Which makes it a bit confusing to choose date/time setting to get specific data. I think I will add a checkbox to convert all the dates to either GMT or Local server time. Then you can choose how you want it to behave and it will make it easier to find specific times. Sound good?

      Delete
    2. Okay, try out Version 1.02 and see what you think.

      Delete
  5. James, appears our users are now getting a different set of options in the Rate My Call screen. Instead of the 5 your tool sees there is a new batch of 8. I can see the new stuff in the db but not in your tool. Have you seen any of this yet?

    ReplyDelete
    Replies
    1. Hi Matt, thanks for the feedback. It appears that the latest Click to Run 2016 version of the client (which is the latest darker blue version of the SfB client UI) also has completely different Rate My Call issue descriptions and Token IDs. I am working on a new version of the script that will also handle the feedback from these clients... :)

      Delete
    2. Hi Matt, I have introduced support for the new S4B client issue types in version 1.04. So give this version a try.

      Delete