Best Practices in
Casino Mystery Shopping
“You can expect what you inspect.”
- W. Edwards Deming
This quote is as true today as it was when the famous quality control engineer and professor W. Edwards Deming coined it. Casino mystery shopping is an excellent tool to inspect and test for the specific service behaviors you expect from guest-facing personnel. Casino mystery shopping presents managers with two benefits: measurement and motivation. Not only does casino and resort mystery shopping test for and measure the presence and frequency of expected behaviors, just the act of mystery shopping, as Deming argued, motivates these behaviors. If employees know what is expected of them and also know they will be evaluated against these expectations, they will increase their compliance to these expected service behaviors.
Beyond measurement and motivation, best in class casino mystery shopping offers an analytical path to evaluate the importance of specific behaviors in terms of advancing the resort’s overall guest experience goals. Not all behaviors are equal - some are more important than others. Best-in-class mystery shopping programs identify the behaviors that matter the most, those which drive guest satisfaction and loyalty.
The guest experience at resorts is not just composed of discreet interactions, but is the result of a collection of interactions across the entire spectrum of touch points. Best-in-class casino mystery shopping programs focus on the guest-resort interface, taking a holistic approach, evaluating the guest-resort interface across multiple touch points.
The foundation of all best-in-class guest experience research is defining objectives. Defining objectives serves as a loadstar for the research, ensuring it stays on course providing useful, relevant research from which clear actions can be taken to improve the guest experience.
Overall Guest Experience Objective
Having an overall objective for the guest experience is perhaps the most important thing for management to consider. Best-in-class brands across all industries have an overall objective to their customer experience and they understand how this objective supports broader corporate goals.
To explore your overall objective for the guest experience, ask yourself: how do you want the guest to think, feel or act as a result of a visit?
For example, do you want the guest to have:
- Increased return intent?
- Increased loyalty?
- Increased purchase intent?
The answer to this question will, again, act like a loadstar by ensuring your casino mystery shopping program will be aligned with your overall corporate objectives. It will provide a critical basis from which to determine the extent to which the overall guest experience supports the brand’s overall business objectives.
Brand Adjectives: After defining your overall business objective for the brand experience, the next step is to consider your desired brand personality. There are a number of ways to define your desired brand personality. In our experience one of the best ways to define is to develop a list of adjectives of how management wants the brand perceived as a result of the guest experience.
Specifically, ask yourself: When guests visit our property, how would we want them to describe our brand, what 5 or 6 adjectives would we use to describe our desired brand personality?
The answers to these questions will produce a list of 5 of 6 attributes which will serve as a mirror to hold up to the guest experience, gauging the extent to which the guest experience supports the desired brand personality.
Brand Statements: Similarly, valuable perspective into the brand personality is achieved by developing a list of statements you would want guests to use to describe your brand. Specifically, ask yourself: what 5 or 6 statements would we like guests to use to describe our brand?
For example, statements such as:
- We are like a trusted friend.
- We are interested in guests as people, not just the bottom line.
- We are committed to the community.
- Our environment is clean and fun.
These statements should then be built into the mystery shop program, asking shoppers the extent to which they agree with each of these statements. Like the brand adjectives, these brand statements will serve as a benchmark from which to compare the guest experience to your brand objectives.
Define Touch Points
After a definition of your desired brand personality, the next decision in designing a casino mystery shop program is to determine the touch points to be evaluated. Again, as we have observed, the guest impression of the overall experience is not made up of individual interactions, but is the result of the constellation of interactions during the stay at the property. Therefore, in deciding what touch points to include in the evaluation, guest experience managers should consider the universe of touch points a guest may encounter.
At the conclusion of this process, you may come up with a list of touch points that looks like this:
After the definition of the desired brand personality, and the selection of the universe of touch points to be evaluated, the next step is to define the behaviors to be tested within the mystery shop.
Fundamentally, best-in-class mystery shopping programs focus on behaviors. There is a role for subjective impressions, which will be discussed later; however, empirical behaviors are the foundation of mystery shopping – indeed, measuring these behaviors is the purpose of mystery shopping.
To construct this list of expected behaviors, go through the list of touch points and define the behaviors you expect from the employee. Ask yourself, for each touch point, what behaviors do you expect?
As an example, you may get a list of questions to ask yourself across each touch point as follows:
What specific behavioral expectations do you have of the Valet?
What specific behavioral expectations do you have of the Server?
Order process behaviors?
Thank you/ parting comment?
Uniform/ Name Tag?
Physical environment of the restaurant?
Food quality attributes?
What specific behaviors do you expect from dealers?
Uniform/ Name Tag?
Thank you/ parting comment?
Best-in-class guest experience programs should seek consistency across touch points. For example, a client of ours has a framework for each touch point and employee to serve as a guide for the guest experience. They call this framework the Seven Totems of the Guest Experience.
Consistency is key to the guest experience. Consistent resorts have more loyal guests. In our experience, clients in the highest quartile of consistent delivery have 20% higher loyalty ratings than clients in the lowest quartile of consistency.
Touch Point Sampling Plan
In the search for consistency across touch points, the next step is determining the distribution of evaluations to each touch point.
Before we address the issue of distributing evaluations across touch points, it is instructive to consider visit types. Typically, most visit types fit into two general categories: overnight and day/evening visits.
The reason it is important to consider visit types is they are the blank canvas upon which to assign touch points. Using overnight and day/evening visit types we can expand these two into three: Overnight (evaluating every possible touch point), and two different day/evening shops (one with a Slots/Keno playing experience, the other with a Table Games/Bingo playing experience). Now we can start distributing evaluations of the above touch points across these three visit types.
For example, assume each visit type will evaluate the following areas:
Day/ Evening (Slots Keno)
Day/ Evening (Table Games/ Bingo)
Now, the next step is to determine the number of monthly visits and calculate the total number of evaluations. Assuming a sample plan of 3 monthly visits (1 overnight and 2 day/evening visits) the total annual evaluations of each touch point are:
Total Annual Evaluations
Once objectives have been defined and touch points selected, questionnaire design is simply the process of mapping each objective to the questionnaire. This is achieved by simply writing a question to capture each of the expected behaviors for each touch point.
In addition to designing a questionnaire to satisfy each behavioral objective within each touch point, it is also a best practice to design consistent elements across all touch points. Consistency of behavioral expectations builds consistency into the entire guest experience. Additionally, we find a lot of value in asking the brand personality questions across all touch points. This approach gives an apples to apples comparison across each touch point, and provides insight into the extent to which each touch point supports the desired brand personality.
Keep it Simple
Questionnaire design should be kept as simple as possible. It should be as complex as necessary to achieve the program objectives - and no more. Unrealistic scenarios and overly complex questionnaires increase the likelihood of shopper misinterpretation, and frustration for all stakeholders in the program: shoppers, mystery shop provider as well as the end client. Keep it simple - simpler designs work better, and provide more value.
Anticipate the Analysis
Finally, all research should anticipate the analysis in the design process to ensure that every element needed to complete the analysis is present. This is best done by identifying and focusing on the desired outcome management wants from the guest experience.
- What is your desired brand personality?
- What adjectives would you want used to describe the brand based on the guest experience?
- What statements would you want guests to agree to regarding the guest experience?
- What is the overall objective of the guest experience - is it return intent?
The answers to these questions will anticipate the analysis and build in mechanisms for Key Driver Analysis to identify which behaviors are more important in driving this desired outcome.
What, How, Why
As we’ve discussed before, best-in-class mystery shop programs focus on observations of empirical behaviors, combined with subjective impressions and comments about the guest experience. Each of these three design elements work together to identify the behaviors which matter most – behaviors which drive the overall guest experience objective – like purchase intent. In consort, these three elements investigate the “what”, “how” and “why” of the guest experience.
As we discussed previously, observations of objective service behaviors are the foundation of best-in-class casino mystery shopping programs. These empirical observations should reflect management’s behavioral service standards, and thus act as a mirror to hold up against the guest experience to determine the extent it reflects these service standards.
Kinēsis prefers to always link guest experience service behaviors to the broader guest experience objectives of each client. If, for example, return intent is a broader objective of the guest experience, we like to ask shoppers if they have been an actual guest, how the experience influenced their intention to return. Later, we will discuss how these are used in the analysis.
Paired with the return intent question, an open-ended follow up question should be asked, asking why the shopper rated their return intent as they did. This gives top-of-mind impressions of the shopper’s experience in their own words – pure feedback from the shopper. Later, we will discuss how this pure feedback is used in call to action analysis.
Reducing a mystery shop down to a numeric score is a common practice. Methodologies for scoring mystery shops vary, however, the common method assigns a point value to every behavior measured, and calculates a numeric score by dividing the points earned by the points possible. This yields a score that is typically expressed by a percentage of points possible. This score can be calculated independently for any business unit (employee, touch point, property, corporate).
Not all behaviors are necessarily equal, some of them are more important than others. Therefore, most scoring methodologies allow for behaviors to be weighted by assigning a higher points possible. Typically, at the start of a program, weights are assigned subjectively according to management’s assignment of points possible. Later, after a statistically valid number of shops is completed, a more objective method of assigning weights using the strength of the association with guest return intent as measure of each behaviors’ relative importance.
Program Launch & Fielding
A property’s employees, particularly guest-facing employees, are the key to a successful guest experience. Best-in-class mystery shop programs ensure that guest-facing employees are properly prepared for the program. In order for a mystery shop program to achieve its potential, employees need to brought into the program. Otherwise internal resistance may distract from the success of the program.
Mystery shopping should not be a surprise to employees shopped. Key to obtaining buy-in is to ensure all employees are fully informed of the program and its objectives, including the specific behaviors shoppers are asked to observe – even to the point of giving employees a copy of the questionnaire. This practice alone will improve employee performance before the first shop is completed. Once shops are delivered employees should be trained on how to read reports and how to most effectively use the information provided, and set improvement goals.
Best-in-class mystery shopping programs provide adequate administration. A strong administrator should be appointed to be the primary point of contact both internally and with the mystery shopping provider. A strong administrator will keep all stakeholders focused on improvement.
Disputed shops are part of the process. Mystery shops are just a snap shot in time, measuring complex service interactions. As a result, there may be extenuating circumstances that need to be addressed, or questions about the quality of the mystery shopper’s performance that require both a fair and firm process to dispute shop scores. Fairness is critical to employee buy-in and morale. Firmness is required to keep the number of shop disputes in check, and cut down on frivolous score disputes.
Call to Action Analysis
Research without a clear to call to action may be interesting, but not very useful. Using the “what”, “how”, and “why” structure of questionnaire design, we build in call to action analysis into study design. Call to action elements include: brand personality, key driver analysis, subjective content analysis, all culminating in gap analysis which identifies areas for improvement with the most potential for return on investment.
As previously discussed, Kinēsis likes to define the desired brand personality using adjectives and agreement with brand statements.
Key Driver Analysis
The overall guest experience must support broader corporate objectives. Previously, we observed that best-in-class guest service organizations always have a broader corporate objective which are served by the guest experience – service isn’t good because it’s good, service is good because it fits into the overall business plan.
For example, a common objective of the guest experience is return intent, to what extent does the guest want to return to the property as a result of the experience.
Key driver analysis is a measure of the importance of each behavior. Grouping shops with positive return intent together and comparing them to shops with negative return intent allows for a behavior by behavior comparison of shops according to their relationship to return intent. In comparing shops with positive return intent to negative, calculate the difference for each behavior between shops with positive return intent and negative return intent. This difference is a measure of the strength of the relationship of the behavior to return intent.
Behaviors with stronger relationships to return intent can be interpreted as key drivers of return intent and are therefore more important.
To add context to the return intent rating, Kinēsis likes to pair this rating scale with an open-ended question designed to elicit why the shopper rated the return intent as they did. These open-ended responses give us subjective un-aided top-of-mind feelings about the guest experience. Mining these open-ended comments for information involves a process of content analysis, where each comment is analyzed for themes they contain. Each theme is coded for quantitative analysis by theme, producing a frequency by theme.
Behavioral Gap Analysis
As noted earlier, empirical observations of behaviors are the backbone of casino mystery shops. It is fitting, therefore, that they play a central role in what is the most useful analysis technique – gap analysis. This technique designed to identify the service behaviors which will have the most return on investment (ROI) in driving your primary guest experience objectives. So, for example, assume guest return intent is the primary objective of the guest experience. The process of gap analysis is relatively simple. First, for each behavior, take the importance value calculated in the Key Driver Analysis and pair it with the frequency in which the behavior is observed in the mystery shops. This frequency is the performance of the behavior.
Plotting the importance and performance of each behavior on a quadrant chart as follows determines the behaviors with the strongest potential for ROI:
Behaviors in Quadrant I have the highest potential for ROI as they have the highest importance and the lowest performance. These behaviors are low hanging fruit, improvement in performance is relatively easy as there is room for improvement, and they are also important in terms of driving return intent. Therefore, they have the strongest potential for ROI in terms of driving return intent.
Acting on Results
Once researchers have identified the key drivers of the guest experience objectives, management has valuable information to act on in terms of designing training and incentives to motivate the service behaviors which drive these guest experience objectives. There are many tools to train and incent these desired behaviors, the two which we will discuss below are: balanced score cards and employee coaching.
Balanced Score Card Approach
Managing the guest experience should take a 360-degree view of the experience using a variety of metrics. Balanced score cards are an excellent tool to get this 360-degree view.
Balanced score cards incorporate four categories of measurement:
- Guest experience metrics (such as guest surveys or mystery shopping);
- Financial metrics (business unit profit and loss);
- Internal business metrics (such as cycle time, employee productivity, or employee satisfaction); as well as
- Innovation and Learning metrics.
Kinēsis recommends that within these four categories managers of the guest experience select specific metrics most relevant to resort goals, such as average daily rate, profit and loss, cycle time, mystery shop and guest satisfaction. A word of caution, however, care should be taken in the selection of these metrics, too many and it will prove too difficult for stakeholders to absorb. Managers should select a few key metrics for the balanced score card of particular significance to the organization.
Guest experience research without mechanisms to improve the guest experience may be interesting, but is unlikely to yield much return on investment. One of these mechanisms for improvement is coaching employees towards improvement. For example, if it is important to monitor suggestive selling, a mechanism should be built into the mystery shop program to identify instances where the employee fails to suggestive sell and target coaching to the employee.
There are a variety of tools available to help managers identify and coach to areas of need. Kinēsis uses an online coaching tool which flags instances requiring coaching and prompts the employee’s manager to coach the employee, as well as providing a tool to log the coaching and produce reports for management to monitor the overall coaching process.
Plan for Change
Best-in-class mystery shopping programs align the behaviors measured in the mystery shop with guest expectations and management service standards. Therefore, a feedback loop should be in place between guest survey research and established service standards to constantly align the mystery shop to guest expectation-based service standards. This informed feedback loop will ensure the behaviors measured are aligned with guest expectations.
Even the best possibly designed mystery shop program requires ongoing adjustments. In our experience, mystery shop scores will cluster together or flatten out over a period of time as employees improve their service behaviors based on the motivational effect of mystery shopping. To keep this natural process of flattening out from diminishing the value of the program, periodic reviews should be conducted with changes as appropriate to keep the mystery shop program relevant and useful.
Best-in-class guest mystery shopping programs take a holistic approach to measuring the guest experience. They start with clearly defined objectives for the guest experience. They measure multiple touch points, with the understanding that the guest experience is comprised of a constellation of service interactions while at the property.
Best-in-class mystery shopping programs focus on objective empirical behaviors. They anticipate the analysis and determine the importance of each of these behaviors in terms of driving the overall objective of the guest experience - such as return intent.
Properly designed, mystery shop programs are not only measurement tools, but motivational tools. The mere act of measuring the service behaviors you expect from staff has a transformative effect on the actual behavior of guest-facing personnel. W. Edwards Deming was right.
You can expect what you inspect.
Eric Larse is co-founder of Seattle-based Kinesis, which helps companies plan and execute their customer experience strategies. Mr. Larse can be reached at firstname.lastname@example.org.
"Beyond measurement and motivation, best in class casino mystery shopping offers an analytical path to evaluate the importance of specific behaviors in terms of advancing the resort’s overall guest experience goals.".