Proving the performance of a new revenue management system
AbstractRevenue management (RM) is a complicated business process that can best be described as control of sales (using prices, restrictions, or capacity), usually using software as a tool to aid decisions. RM software can play a mere informative role, supplying analysts with formatted and summarized data who use it to make control decisions (setting a price or allocating capacity for a price point), or, play a deeper role, automating the decisions process completely, at the other extreme. The RM models and algorithms in the academic literature by and large concentrate on the latter, completely automated, level of functionality. A firm considering using a new RM model or RM system needs to evaluate its performance. Academic papers justify the performance of their models using simulations, where customer booking requests are simulated according to some process and model, and the revenue perfor- mance of the algorithm compared to an alternate set of algorithms. Such simulations, while an accepted part of the academic literature, and indeed providing research insight, often lack credibility with management. Even methodologically, they are usually awed, as the simula- tions only test \within-model" performance, and say nothing as to the appropriateness of the model in the first place. Even simulations that test against alternate models or competition are limited by their inherent necessity on fixing some model as the universe for their testing. These problems are exacerbated with RM models that attempt to model customer purchase behav- ior or competition, as the right models for competitive actions or customer purchases remain somewhat of a mystery, or at least with no consensus on their validity. How then to validate a model? Putting it another way, we want to show that a particular model or algorithm is the cause of a certain improvement to the RM process compared to the existing process. We take care to emphasize that we want to prove the said model as the cause of performance, and to compare against a (incumbent) process rather than against an alternate model. In this paper we describe a \live" testing experiment that we conducted at Iberia Airlines on a set of flights. A set of competing algorithms control a set of flights during adjacent weeks, and their behavior and results are observed over a relatively long period of time (9 months). In parallel, a group of control flights were managed using the traditional mix of manual and algorithmic control (incumbent system). Such \sandbox" testing, while common at many large internet search and e-commerce companies is relatively rare in the revenue management area. Sandbox testing has an undisputable model of customer behavior but the experimental design and analysis of results is less clear. In this paper we describe the philosophy behind the experiment, the organizational challenges, the design and setup of the experiment, and outline the analysis of the results. This paper is a complement to a (more technical) related paper that describes the econometrics and statistical analysis of the results.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by Department of Economics and Business, Universitat Pompeu Fabra in its series Economics Working Papers with number 1196.
Date of creation: Nov 2009
Date of revision: Jan 2010
Contact details of provider:
Web page: http://www.econ.upf.edu/
Revenue management; airlines; sandbox testing; econometric analysis.;
Find related papers by JEL classification:
- M11 - Business Administration and Business Economics; Marketing; Accounting - - Business Administration - - - Production Management
- M31 - Business Administration and Business Economics; Marketing; Accounting - - Marketing and Advertising - - - Marketing
- L93 - Industrial Organization - - Industry Studies: Transportation and Utilities - - - Air Transportation
This paper has been announced in the following NEP Reports:
- NEP-ALL-2010-04-04 (All new papers)
- NEP-CMP-2010-04-04 (Computational Economics)
- NEP-MIC-2010-04-04 (Microeconomics)
You can help add them by filling out this form.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ().
If references are entirely missing, you can add them using this form.