Measuring success of a meeting or event is a must in today’s market. The first step, of course, is determining your goals and objectives for your meeting or event. This article talks about measuring, its importance and how to make it manageable.
By: Ira Kerns,Managing Director, MeetingMetrics.
You’ll recognize these acronyms as attempts to define whether or not a meeting has been effective. Has it satisfied the specific needs of both meeting owners and meeting attendees (customers)? How do these measurement methods work? Which are easier or more difficult to implement in the real world of a meeting professional trying to determine how well a meeting performed? Do some apply only to certain types of meetings?
The following is my attempt to provide an overview without getting into boring nitty-gritty.
First, it is important to point out that while there is a good deal of overlap among them, each has a specific point of view about what is central to the measurement process.
ROO (Return on Objective)
Return on Objective focuses on defining and achieving specific measurable objectives. It is a basic management approach, which can be applied to almost every aspect of business and has been around for almost half a century.
In its classic form, a set of objectives is created, each of which contains within itself a numeric measurement criteria. Here are some examples:
- “Confidence levels regarding our marketing program among attendees will be on average at least 80 percent after the meeting.”
- “Pre-meeting communications and promotions will stimulate increased meeting attendance of at least 15 percent.”
- “Meeting participants will have gained at least one new skill as a result of attending the meeting.”Objectives may be set at several different levels:
• Global-level strategic objectives may be set for the meeting overall.
• Objectives may be set for individual sections of the meeting.
• Objectives may be set for specific meeting presentations, sessions, and activities.
• Objectives may be set for each segment of the meeting population.Each of these can be developed and measured, depending upon how detailed you want to get with your event measurement. Certainly for your largest and most important event, you would want to measure the results and impacts in greater detail and depth than for other less important meetings.
For ROO to work, very specific and measurable objectives must be defined and written for every desired outcome of the meeting well in advance of the meeting and must be approved by the meeting owner group. Then meeting content and activities are designed to accomplish the objectives.
To determine the objectives, some planners will use interviews, focus groups, and surveys to surface issues; however, well-organized and focused pre-meeting research often is not done. Instead, this critical task is left to informal, anecdotal conversations with some of the meeting owners, with the assumption that they know what is needed and wanted by both leaders and attendees (customers).
The meeting agenda is essentially a collection of designed education and social experiences to address each of the stated objectives that will be measured after the meeting. Measurement is most often accomplished using an online, post-meeting survey questionnaire.
ROI (Return on Investment)
ROI is a standard, well-accepted business KPI (key performance indicator) that is used to define the financial return from a business activity, usually a project, program, or even an entire business operation in which a financial investment is made. The ROI is arrived at by adding up the associated costs, monetizing (i.e., converting into financial terms) the accumulated benefits, and subtracting the costs from the benefits to arrive at the ROI.
This is expressed as a percentage of the costs. So an ROI of 100 percent would be break-even as your return would equal your costs. A more desirable ROI would be something greater than 100 percent; for example, 125 percent, which would be an ROI of 25 percent after subtracting all your costs.
The ROI Institute has developed a method of measuring the ROI of a meeting by way of a five-level “value chain” framework, which measures both tangibles (results that can be quantified and monetized) and intangibles (results that are too difficult or impossible to monetize). Only the tangibles are used to calculate the ROI as a financial return measure. Each step on the ladder is essential in terms of its additive contribution in achieving the overall ROI outcome.
Five Levels of Evaluation*
Level 1 Reaction and Planned Action
Level 2 Learning
Level 3 Application and Implementation
Level 4 Impact (Tangibles and Intangibles)
Level 5 Return on Investment
*© ROI Institute
Corporate Sales Meetings ROI Chart
An ROI study of a meeting usually does not take place until two or three months or longer after a meeting as time must pass for results from the meeting (tangible and intangible) to accumulate before the study can be undertaken.
To apply the ROI methodology to a meeting, one has to attend one or more ROI Institute workshops (3–5 days) to learn the methodology, followed by the preparation of an ROI Study for an actual large group meeting. The report is then submitted to the ROI Institute for review and feedback to assure it was done correctly.
Alternatively, the ROI Institute will conduct ROI studies of meetings for a fee and/or recommend certified ROI consultants to provide the service. Find more information at the institute’s Web site. www.roiinstitute.net
ROE (Return on Event)
MeetingMetrics was founded more than a decade ago to incorporate another measurement methodology—ROE. I developed and used this measurement with many companies as a consultant, creating a specialized, professional-level online tool-set of survey templates and reports for use by meeting professionals.
ROE is a pre-meeting/post-meeting measurement method for both optimizing the results of a meeting and for accurately measuring those results. This can be done with as few as two or as many as three or four surveys.
A pre-meeting, qualitative survey (“Discovery”) is often used, followed by a pre-meeting, quantitative survey (“Baseline”) a couple of months or so before the meeting. The purpose of this research is to surface and define the needs of your meeting attendees (customers) and meeting owners.
The Discovery survey is relatively short, using 8–10 open-ended questions that capture participants’ needs and views in their own words. It uses a small sample of 50–100 people from the larger body of prospective attendees.
The Baseline survey is a substantially longer, census survey (all prospective attendees are invited) that builds on the knowledge gained from the Discovery survey results. It asks about (and measures using rating questions) specific opinions, interest levels, importance, priorities, etc.—all the important aspects of the meeting.
Survey findings are analyzed and a report provided to the meeting design team for them to develop a set of specific meeting objectives and a meeting plan that addresses the identified needs of the key segments of meeting owners and meeting customers who will attend.
A post-meeting evaluation survey (“Benchmark”) is conducted within a week or so after the meeting. The summary report includes a comparison of pre-meeting and post-meeting benchmark question results detailing the ROE findings that explain the impact of the meeting on the attendees overall in terms of specific positive or negative shifts in knowledge and skills, perceptions, attitudes, motivation, commitment, and intentions as a result of their meeting experience.
It also rates the value and usefulness of key meeting elements and asks about attendees’ post-conference needs.
ROE Sample of Pre-meeting/Post-meeting Benchmark Results
A followup post-meeting survey (“Tracking”) is recommended a month or two after the meeting for tracking attendees’ post-meeting views, actual behaviors and actions, business impacts, and accomplishments specifically related to the meeting objectives. ROE measures the following:
- Business Results/Impacts
These three measurement approaches have been used for years with success, but with the advent of Web, mobile, and virtual event technologies, even more measurement options are out there for meeting planners to explore.
For example, for those meetings with a primary focus on training and professional education, specialized online evaluation and test questionnaires with on-demand scorecard and CE certificate services are available from my firm and other sources.
In addition, measuring attendee engagement at meetings and events is becoming an important priority. I plan to address both measurement for education events and engagement in future articles.