Book a demo


Blog Detail Title

Short Content Here

by Author Name

Date Published

Knowledge, App Partners

How to validate an RMS

Edward Neale October 4, 2021

When establishing - or improving - the Revenue Management System (RMS) setup of a hotel, there are many considerations that need to be taken into account. Pace Revenue have distilled one of these key concerns: how to validate an RMS and make it work for you. This blog piece was previously posted on Pace Revenue's own blog as How to fix the validation problem when choosing an RMS.

If you're reading this article, you're already taking the first step toward improving your property's RMS. If you find yourself asking "where do I start with the RMS journey?", then skip ahead to this article: How to Choose an RMS. Once you've figured out which RMS you'll choose, you'll be well equipped for this article about the next step: validating the RMS.

Fact: before you can even install your Revenue Management System (RMS), you need to validate it. The problem is that there is no commercial tool to help revenue or property managers with validation, and there is little shared know-how. There is no standardized methodology to even begin to implement one. In reality, what does exist is a range of imperfect comparisons performed ad hoc. 

Ultimately, hoteliers need to figure out what the return on investment (ROI) for any given RMS is. So far, only big corporate chains with deep pockets have been able to tackle the conundrum. But for smaller chains and independents, the sensational marketing of revenue management vendors too often dictates the decision-making process. Plenty of flashy numbers are plastered all over RMS sales pitches. They look exciting. They are aggressive. The system is always 'the best system ever!'

So an executive team gets sold on the idea and signs off on all the contracts. No testing. No validation. Now a lack of trust in validation has led to the RM vendor space becoming a horse trading bazaar of sorts.

For example, a hotelier may decide to choose a ‘cheaper’ RMS package because they think they are saving money but, in fact, they are overlooking the fact that performance is EVERYTHING. A difference in revenue gains of 1-2% between one RMS and another is HUGE - so much so that the price paid for the system itself becomes almost irrelevant. Mathematically, you should pay more for the RMS that gets you the better performance, even if it’s just a 1% boost. The better ROI will compensate for the extra cost. Every time.

“ A difference in revenue gains of one or two percent between one RMS and another is a HUGE difference, so much so that the price paid for the system itself becomes almost irrelevant”

Smaller brands don’t have the scale or the resources and too often ignore this statistical truism, even though they shouldn’t. Ironically, it’s the smaller groups that will benefit the most from implementing an RMS. Most hotels still do not use an RMS at all and it’s the pricing capability of the solution which has the biggest impact, rather than the inventory controls and other trimmings championed by many software companies. It all starts with actually being able to determine what the performance of an RMS actually is and hoteliers need an affordable tool to help with that.

Field experiments are everything

To properly validate an RMS, you need to do robust A/B testing or, in the academic parlance, conduct ‘field experimentation’. Field experiments are studies that take place in a controlled environment, similar to the natural setting. Take the timely example of testing a vaccine for efficacy; for obvious reasons, you don’t just simply want to give the vaccine out to a bunch of people to test whether or not it works - this is just not how a proper trial works. You need to perform the test with specific variables and even go so far as to give some people placebos. Now consider the RMS as the vaccine - it’s not going to be helpful unless it is based on good data.

Let’s agree that you need to test different revenue strategies in the field. Familiar brands like Amazon, Uber, and Netflix, run these experiments all the time. Even AirBnB regularly conducts A/B testing. Meanwhile, the vast majority of hotels never test their revenue strategies in the real world. Field experimentation is possible and must start being done more regularly.

Hoteliers are already testing

Very few field experiments have been done in hospitality for revenue management and the little that has been done has not been well documented publicly. There’s one here and another one here.  

There is still hope! At the corporate chain end of the hospitality industry - e.g. the Hiltons and Marriotts - the managers will always do a controlled experiment to validate an RMS before they upgrade or switch. The problem is that this involves hiring teams of data scientists who run statistical appraisals over several months. Since a DIY tool doesn’t currently exist, what ends up happening is that these brands hire an entire consulting team of ten people to run the tests internally.

Without the same resources, smaller hotels are left to run crude tests on the efficacy of an RMS themselves. They may use RevPAR (revenue per available room), but this is not ideal because it could include market impacts. Perhaps they go a step further to remove any influence the market may have by looking at the revenue generation index (RGI). Once again, this is not completely accurate because the numbers could be unfairly influenced by a competitor’s new refurbishment or their new marketing initiatives. 

A promising reality is that marketing departments of hotels are already doing field experiments. CRM systems often use them: serving customers different emails to see what garners the best reaction is - technically - a field experiment. If hotels are already used to doing A/B testing on their ancillary pricing, promotions or marketing, then they are more likely to demand it of their RMS providers in the future. However, field experimentation is not something near the top of what is required when choosing an RMS, although it definitely should be!

What we would like to see

The entire hospitality industry should have access to the know-how and have access to affordable tools to independently validate the performance of a revenue management strategy. Currently, we do not even have a clear way within the hospitality industry to do an ROI calculation for an RMS. This can change, and the two biggest change factors in our control are:

  1. Share the knowledge!
    Big corporate groups should publish their experiments as it would lead to more know-how in the RM community. What can smaller hotels and properties learn from these market leaders in field experimentation?
  2. Build a tool
    This would mean you could validate an RMS affordably with perhaps one data scientist, instead of a larger hotel’s typical internal team of data scientists, IT guys, and project managers.

There are challenges, certainly, but it’s more than possible.

How to build a tool

What kind of designs and strategies do we imagine a good tool would incorporate? To get the ball rolling, we designed three complementary ways to start testing any given revenue management strategy.

  1. Property Splits. This is what the big chains do when they run RM tests and the rough idea is that you put one half of your properties on one strategy, and the other half on another in order to test the results. It’s a good strategy for looking at a whole group but it’s not so granular, and for smaller groups and individual properties it’s not that relevant.
  2. Alternating Periods. We can start to get a bit more granular and look at the property itself by running different strategies on different reservation nights. You can also combine it with multiple periods. What should the length of the test period be? One week? Three months? Seasonality effects are an issue but you can control for that.
  3. High Frequency Price Updates. This one is the most complex to pull off but it allows for much more granularity, as you can start to look at single stay nights. In short, you could allocate half of your inventory to one algorithmic strategy and the other half to another strategy, which allows you to answer more granular questions and you can change it rapidly (hourly).


Looking to the future

So what can we do today to prepare for this future? This whitepaper is a start because vendors don’t currently offer genuine experimentation systems. But there are some hacks…

The first design - Property Splits - could be implemented manually and an RMS could help with the data visualization through custom-built tools. In terms of analytics, one would need to build custom dashboards targeting experiment design and analysis. That’s where a customizable business intelligence (BI) tool comes into play.

We could start with hotels that have the right PMS (cloud, two-way integrations etc.) and provide some technical solutions that will help to reduce the cost so that hoteliers can start validating their revenue management strategy.

Hoteliers can also play their part by adding an initial step to their RMS shopping process - demanding validation from their vendors and asking for help with that validation. We need to work together and build a solution that helps push the revenue management industry forward and putting some fire under our toes will help.

However, we need to go beyond customizable BI to a future where these kinds of dashboards are an industry standard and accepted throughout the industry. Wider adoption and bigger changes will take several years but we must start now as, ultimately, it will be transformational for hospitality. It boils down to what the return on investment from a given RMS is. Because if the ROI is 5-6% then the price tag shouldn’t matter.

For further information on the topic, continue reading Pace Revenue's free article How to Choose an RMS or schedule a call with both Pace Revenue and apaleo to learn how we can help you improve your RMS setup. 

pace-revenue-mockup-square (2)


Edward Neale

Posted by

Edward Neale
Marketing Director at Pace Revenue


by Juliana Hahn

Apr 14, 2022 8:45:00 AM

Rapid Roll Out: Go live with your PMS in days, not months

Here at apaleo, we understand that adopting new tech can feel like a terrible hassle. Weeks of research, discovery calls, demos and negotiations take up a lot ...

by Lex Parree

Aug 19, 2021 12:00:53 PM

Why Streaming is Important for Your Hotel Guests

So much can be done with a hotel's In-Room-Entertainment (IRE) services, from direct communication with guests to pure entertainment. However, technology ...


Subscribe to thepulse newsletter:

  • The coolest industry stories and research
  • Sweet innovations from our app community
  • Secret invites and special offers
By entering your email you expressly consent to letting us use your email address to send you our newsletter and other relevant stuff.