Client Satisfaction Research: Service Dimensions and Drivers of Overall Satisfaction

Table of contents

The section presents the results of the multivariate analysis that was undertaken with selected survey findings in order to determine the drivers of satisfaction. The analysis revealed three main service dimensions as drivers of overall satisfaction:

  1. Knowledge and Information: Skills of the staff and the system to provide accurate information to the clients as or before required.
  2. People Skills: Softer skills required by Service Canada staff to serve the clients.
  3. Access Speed: The amount of time it took to get the service and accessibility of the service.

Overall, these service factors explain 53% of the variance in overall satisfaction among clients of Service Canada. This is a highly acceptable regression "fit", meaning that these factors, indeed, are highly correlated with overall satisfaction. Of the three factors, Knowledge and Information, was the most important service dimension, followed closely by People Skills and Access Speed.

A detailed discussion of the multivariate analysis follows below, including the corresponding statistical tables.

1. Overall Service Dimensions

Initially, regression models were tested using all of the core service dimensions (Q9 to Q21) to predict overall client satisfaction. Due to the large number of highly interrelated variables, the model coefficients for each variable tended to be small and therefore challenging to interpret. To avoid the multi-collinearity problem (i.e. high correlation among the predictor variables in the regression equation), the possibility of using factor analysis to reduce the 13 variables into a smaller set of composite measures was tested.

Using normal cut-offs for the number of factors yielded only one factor, which demonstrated that all of these variables are highly interrelated and could be used to create an overall client satisfaction index. The exceptions were the variables for protection of personal information (Question 18), service in the official language of choice (Question 12, and courteousness of staff (Question 14). These variables did not load highly on the single factor and did not correlate as well with overall client satisfaction. As a result, Questions 12, 14 and 18 were removed from the factor analysis.

The subsequent analysis showed that these variables added little value for predicting client satisfaction and so were excluded from the regression models. This does not mean that these dimensions of service are not important to maintain high levels of client satisfaction. It simply means that, currently, the level of service provided may be sufficiently high that there is limited variation in these measures, which then limited our ability to measure correlations with overall client satisfaction. A change in the approach to service that had an impact on these service dimensions could have very negative impacts on overall client satisfaction.

To see if forcing a larger number of solutions on the factor analysis would reveal any grouping of variables that could be used to create distinct composite measures, three to four solutions were initially tested using all of the variables. The most sensible results were obtained using a three-factor solution. Based on the loadings of the variables on these factors, the three factors were: Knowledge & Information, People Skills and Access Speed (including accessibility and timeliness). Table 3.1 (below) shows the service dimensions associated with each of these factors. The Knowledge & Information factor represents the skills of the staff and the system to provide accurate information to the clients as or before required. The People Skills factor represents the softer skills required by Service Canada staff to serve the clients. The last factor, Access Speed, is represented by two variables, the amount of time it took to get the service and accessibility of the service.

Reliability of these factors was tested using Cronbach's Alpha, which confirmed the inclusion of these variables in the three factors. For the subsequent regression analysis, the factors were created using a simple linear composite by adding the variables in each factor. This composite index was converted to a five-point scale similar to the original variables by dividing the total score by the number of variables used to create the index.

Table 3.1 Client Satisfaction Service Factors

1. Knowledge & Information 2. People Skills 3. Access Speed

Staff were knowledgeable and competent

The information you received was clear and easy to understand.

You were informed of everything you had to do in order to get the service.

It was clear what you could do if you had a problem or question.

Staff gave you what you needed or guided you to others who could help you.

Staff were responsive to your needs.

You were treated fairly.

Staff went the extra mile to make sure you got what you needed.

You were satisfied with the amount of time it took to get the service.

You were satisfied with the accessibility of the service.

2. Overall Client Satisfaction

Table 3.2 (next page) provides a summary of the relationship between overall client satisfaction and the client satisfaction service factors described previously. The second column in the table shows the regression model coefficients for the model using the three client satisfaction service factors as predictors of overall client satisfaction. The third column in this table provides the average rating for overall client satisfaction and the service factors. The fourth column presents the simple Pearson correlation coefficientFootnote 14 that provides a bivariate measure of the direction and strength of the relationship between overall client satisfaction and the client satisfaction service dimensions (ranges from -1 to +1).

The adjusted R2 for the model was .53, indicating that 53% of the variance in overall client satisfaction could be explained by these three service factors. Based on model coefficients, the most important service dimension was Knowledge & Information, with a coefficient of .33. In other words, if the average rating of the quality of the staff or the quality of the information increased by 1, we would expect the overall client satisfaction rating to increase by .33. While changes in an average rating of .33 may seem small, it represents 33% of the clients increasing their overall satisfaction rating by one full rating point on the 5-point scale. This would be the equivalent to 1,323 out of 4,009 clients moving from a neutral rating of '3' to a satisfied rating of '4' in terms of their overall satisfaction with the service provided.

The coefficients for People Skills and Access Speed were only slightly smaller than the coefficient for Knowledge & Information, at .29 and .27 respectively. This finding highlights the fact that overall satisfaction is driven by multiple factors, that failure to perform well on any one of these client service factors could result in a significant deterioration of client satisfaction ratings. In fact, tests using each individual variable in a model rather than the overall factors (Table 3.3 below) showed Q20: 'You were satisfied with the amount of time it took to get the service' and Q11: 'You were treated fairly' had the largest correlations. The Knowledge & Information-related variables were still important predictors of overall client satisfaction; however, these impacts were distributed over a larger number of variables.

Table 3.2 Results of Multivariate Modeling: Service Factors and Overall Satisfaction

Overall Satisfaction and Service Factors Coefficient b (Unstandardized) Mean RatingFootnote 15 Correlation with Overall
*p<.05,**p<.01,***p<.001
Overall Satisfaction --- 4.24 ---
Knowledge & Information 0.327*** 4.28 0.679***
People Skills 0.292*** 4.30 0.664***
Access Speed 0.265*** 4.12 0.645***

Table 3.3 Results of Multivariate Modeling: Individual Service Dimensions and Overall Satisfaction

Overall Satisfaction and Individual Service Dimensions Coefficient b (Unstandardized) Mean Rating
*p<.05,**p<.01,***p<.001
Overall Satisfaction --- 4.24
Staff were knowledgeable and competent .052*** 4.32
Staff were responsive to your needs .088** 4.35
You were treated fairly .137*** 4.50
The information you received was clear and easy to understand .062*** 4.25
You were informed of everything you had to do in order to get the service .081*** 4.31
Staff gave you what you needed or guided you to others who could help you .094*** 4.34
It was clear what you could do if you had a problem or question .054** 4.15
Staff went the extra mile to make sure you got what you needed .065*** 4.05
You were satisfied with the amount of time it took to get the service .170*** 3.98
You were satisfied with the accessibility of the service .083*** 4.26

3. Program/Services

The overall client satisfaction rating was asked relative to a specific service offering the client had experienced (e.g. EI, CPP, OAS, SIN, etc.). Table 3.4 (below) shows the results when the same model was estimated segmented by the service offering the client was asked about. The associated mean scores are shown in Table 3.5 (also below). The coefficients for three factors varied substantially across the services received by the clients, including:

  • Knowledge & Information was dominant for 1-800 (.60) and had the largest coefficient for SIN, EI and AIG/ACG. Knowledge and Information was not statistically significant for WEPP.
  • People Skills was dominant for WEPP (.53) and CPPD (.46) and had the largest coefficient for CPP. People Skills had the smallest coefficient for SIN, and a relatively small coefficient for OAS/GIS and AIG/ACG.
  • Access Speed did not dominate any one service, but had a large coefficient for SIN, OAS/GIS (the largest coefficient for that service), WEPP, EI and AIG/ACG. The coefficient for Access Speed was relatively small for CPP and statistically non-significant for CPPD and 1-800.

Table 3.4 Results of Multivariate Modeling: Service Dimensions and Overall Satisfaction Segmented by Services

Services 1. Knowledge & Information 2. People Skills 3. Access Speed
*p<.05,**p<.01,***p<.001
CPP .250** .366*** .171**
CPPD .276** .460*** .074
OAS/GIS .285*** .200** .346***
EI .358*** .247*** .306***
WEPP .178 .530*** .337***
AIG/ACG .347*** .216** .301***
SIN .395*** .137* .377***
1 800 .604*** .311*** .087

Table 3.5 Mean Service Dimension and Overall Satisfaction Ratings Segmented by Services

Services Overall 1. Knowledge & Information 2. People Skills 3. Access Speed
CPP 4.25 4.20 4.28 4.11
CPPD 4.27 4.23 4.39 4.17
OAS/GIS 4.34 4.38 4.49 4.21
EI 4.16 4.23 4.24 4.03
WEPP 3.88 4.16 4.28 3.94
AIG/ ACG 4.28 4.21 4.28 4.07
SIN 4.27 4.35 4.42 4.11
1 800 4.46 4.51 4.63 4.39

4. Client Segments

Modelling results for selected client segments is provided in Table 3.6 (below) and the associated mean scores are shown in Table 3.7 (below). The key drivers of overall satisfaction for the client segments were as follows:

  • For youth, knowledge and information had the strongest impact on overall satisfaction. The other two factors were statistically significant but smaller in terms of impact.
  • For baby boomers and persons with disabilities, people skills have a bigger impact than the other two factors.
  • For Aboriginal Canada, access/speed and knowledge and information are critically important factors.
  • For men, women, working age adults, seniors and Newcomers, all three factors were fairly similar in magnitude.

Table 3.6 Results of Multivariate Modeling: Service Dimensions and Overall Satisfaction Segmented by Client Type

Service Dimensions Youth (18-30) Working Age (18-64) Seniors (60 plus) Boomers Aboriginal PWD Newcomers Male Female
*p<.05,**p<.01,***p<.001
Knowledge & Information .445*** .324*** .278*** .136* .406*** .182** .307** .316*** .342***
People Skills .222*** .305*** .308*** .436*** .021 .445*** .272* .298*** .280***
Access Speed .289*** .279*** .239*** .288*** .448*** .214*** .265*** .273*** .255***

Table 3.7 Mean Service Dimension and Overall Satisfaction Ratings Segmented by Client Type

Service Dimensions Youth (18-30) Working Age (18-64) Seniors (60 plus) Boomers AboriginalFootnote 16 PWDFootnote 17 NewcomersFootnote 18 Male Female
Overall Satisfaction 4.26 4.24 4.31 4.25 4.17 4.21 4.35 4.21 4.29
Knowledge & Information 4.32 4.29 4.29 4.30 4.16 4.17 4.45 4.23 4.35
People Skills 4.36 4.37 4.40 4.38 4.20 4.30 4.53 4.31 4.44
Access Speed 4.08 4.11 4.18 4.13 3.87 4.03 4.29 4.10 4.15

Footnotes

Footnote 14

It should be noted that, for all three factors, the correlation between the factor and overall client satisfaction was larger than the correlation between overall client satisfaction and the individual variables included in the factor. That is, the composite measure was a better predictor of overall client satisfaction than any one variable in the factor.

Return to footnote 14 referrer

Footnote 15

For the purposes of the multivariate modelling, the unweighted data was used in order to introduce more variability in the data (avoids having the models to explain client satisfaction being determined by the largest programs).

Return to footnote 15 referrer

Footnote 16

Analysis based on only 183 respondents.

Return to footnote 16 referrer

Footnote 17

PWD stands for persons with disabilities.

Return to footnote 17 referrer

Footnote 18

Analysis based on only 257 respondents.

Return to footnote 18 referrer

Table of contents