Problem: UK Visas and Immigration (UKVI) requires the regular production of visa and immigration applications forecasts to assist casework and delivery planning decisions. Forecasts are required by visa type for applications made within the UK and also for applications made at overseas embassies and high commissions. Long range forecasts which contributed to strategic planning were being produced biannually, but these were not considered suitable for operational planning as the accuracy of any forecast will decline as the time since the forecast was made increases.

We were asked to develop a short-term forecasting (STF) approach which would make the best use of the latest information available to produce weekly forecasts. The primary requirement of the solution was that it be as accurate as possible in order to guide decision-making, with the further requirement that individual business units should be able to forecast their own areas of interest with minimal calls upon central resources. This meant that the solution needed to be sufficiently complex to handle all eventualities yet be easy to use.

Approach: The forecasting methodology we developed is time series based so it assumes patterns observed in the past will continue into the future, although users are given the ability to incorporate business assumptions in order to modify the base forecasts. The process uses Excel and VBA macros to provide user friendly interaction with the core forecasting routine which is performed by SAS software. Whilst the existing forecast method utilises monthly data we elected to forecast at the daily level. The daily forecasts could then be aggregated to weekly or monthly level in order to meet individual user requirements.

Methodology: Forecasting is performed by using regression equations to identify the significance and the size of those events that affect the observed demand. Events that were found to be of importance included the day of the week, the week or month of the year and end of year effects (demand often increases prior to higher prices starting in the new year).

Public holidays and other customer centre closures also affect the demand pattern. As each country has its own national holidays, it was necessary to populate an events calendar which holds the dates of importance for all the major countries. Outside of the UK there was a requirement to identify these dates from within the data itself, rather than directly relying on lists of holiday dates as the importance of the various holidays tends to change over time. This was achieved by devising pattern matching rules which aim to distinguish between forced zero demand (due to holidays and other closures) and naturally occurring zero demand on any given day. Holidays were then separated from other closures by consulting the lists of holiday dates by country. There was also a requirement to identify which countries have non-standard weekend days from within the data itself as the Visa centre opening hours may or may not follow local customs.

The inclusion of prediction intervals alongside the standard point forecasts is an important part of the forecasting process as they provide a measure of the uncertainty surrounding the forecasts. In technical terms, prediction intervals differ from confidence intervals in that the former are associated with a random variable yet to be observed, whereas the latter is associated with a non-random but unknown parameter value. If the forecasts the system was providing were considered to be accurate, based on past performance, then they would have a narrower interval band compared to forecasts that were considered to be less accurate. The provision of upper and lower bounds allows the decision-makers to create sensible contingency plans should the true demand be higher or lower than expected.

We also developed an automated accuracy tracker to monitor the forecasting process against actual demand to ensure the forecasting system remains in control. If the tracker value becomes too large then it is an indication that the demand pattern has changed or that forecast updating is not responding quickly enough to recent changes in demand. The number of asylum applications can quickly change by country depending on global events, for example. Providing this tracker information helps the users decide where their attention is best directed.

Outcome: We developed an automatic forecasting tool that has the ability to generate accurate forecasts on a regular basis. Automatic systems using a time series approach have the advantage that they can quickly provide forecasts across a large number of series. However, they do have the weakness that they assume the future will follow the patterns of the past. In order to alleviate this weakness we developed an enhanced system which would identify when the forecasts were less accurate than they should be and make it as easy as possible for the users to apply expert judgement and adjust the forecasts in a controlled manner. Any adjustments are entered as overlays to the otherwise automatic forecast outputs.

The system is currently being pilot tested within a limited number of business units prior to full rollout. The pilot testing will ensure that the user interface is easy to use, the forecasts are sufficiently accurate for decision-making and that users are confident in their ability to use the tool. It will also allow the users to feed back any changes or enhancements that will help them perform their work.

We believe that the system we have developed will prove itself to have wider application, and that it will eventually replace the long range forecasting system it was originally intended to work alongside. This will offer additional benefits in terms of reducing the number of systems UKVI staff need to understand, use and maintain.

Client comment: The client commented, “Andrew has been integral to the development of a short term forecasting tool for the Home Office allowing the production of daily and weekly demand forecasts by the business.  Andrew’s expertise has allowed him to develop a very flexible tool to serve a diverse range of business needs whilst catering for unique and area specific needs.  He has been responsive to the developing demands of the project, building a user friendly, high quality expandable tool, with the necessary documentation and training materials alongside it.” – Howard Birks, Decision Support for Operations, May 2015.

Experience gained: We found that coming into an organisation as an outsider offered a fresh perspective on what can be achieved.  For example, the source data for the forecasting process had always been obtained as monthly data, but by asking questions we found that it could be readily obtained as daily data.  This change offers more insight into the demand patterns and provides greater flexibility in how the data is used.

Throughout our involvement on the STF project, we endeavoured to exceed the customer’s expectations; when they indicated that quarterly updates to the forecasts and views up to six months’ ahead would be sufficient, we set out to deliver a system that could be updated weekly and provide views at least twelve months ahead – at no additional cost.

Contact: Andrew Eaves