Skip to main content
School of Electronic Engineering and Computer Science

Measuring Actual Experience to Improve Networked Applications and Services

 Networks Research Group

 

Researchers at QM have developed a way of assessing how well network based services perform in terms of human experience. This measure, perceptual quality (PQ), is a new technique that identifies reasons for impairments that people notice when using applications.
dashboard_thumb

 

 

 

dashboard_thumbActual Experience (AE), which was formed in 2009 as a spinout of QM, has successfully brought this analytical approach to market. AE's vision is that for organisations of any size and for broadband users at home, the only thing that really matters is whether they are hindered or enabled by the services they consume. AE improves the human experience of applications delivered across global Digital Supply-Chains – allowing people and businesses to work and interact more effectively.

The importance of the global digital economy is indisputable. When people are hindered by IT there is a significant economic and social impact – consumers abandon transactions; individuals and communities are less able to interact and businesses lose revenue as efficiency and morale are affected.

But when IT acts as an enabling technology, the benefits are even more profound – the full economic potential of the multi-trillion dollar Internet economy is unlocked. People and communities are connected and able to consume all that the digital world has to offer – ways to learn, collaborate and engage that transcend national and cultural boundaries.

AE's Analytics-as-Service product takes measurements across global Digital Supply-Chains – gathering information specific to the locations and applications used by AE's customers. These measurements are passed to the Analytics Engine – the result of more than 10 years of research at QM, and the result of AE's intellectual property in both computational algorithms and measurement systems. The Engine performs the complex tasks required to transform simple measurements into meaningful business information:

    Understanding whether staff and customers are enabled or hindered by IT;
    How to mend customer and staff experience;
    How to de-risk business transformations;
    How to manage cost without reducing experience.

AE works with customers ranging from global blue chips to governmental organisations, from large enterprise to service providers, and also provides a free service for hundreds of home broadband users.

AE's unique technology benchmarks how good the digital experience should be, quantifies how good it actually is, and continuously triages the Digital Supply-Chain to identify the reasons for the gap between the two. This enables businesses to locate the underperformance that people notice, helping them focus their support resources where they will have the greatest impact.

IT is a huge investment: 3-8% of top line, according to Gartner. And it’s made for one reason – to enable staff and customers to work more effectively. But does that investment represent money well spent? What happens when the lights are green but customers still complain? How can transformations be de-risked, and how is the most extracted from any investment? These are all supply-chain management questions that remain unanswered in the world of Digital Supply-Chains, until now.

When everything appears to be fine, but customers and staff are still complaining, the Digital Supply-Chain isn’t working properly and the business isn’t enabled, it’s hindered.

With business transformations, new Digital Supply-Chains are being built: new networks, datacentres, applications… Can the CIO be confident that their new Digital Supply-Chains won’t harm the business? Failure to meet business KPIs risks brand, operations and efficiency.

Every part of the Digital Supply-Chain seems to require continuous investment. But CIOs have finite resources – how do they know where to invest where customers and staff will notice most?

Recognised by analysts as an emerging new sector concerned with the human impact of IT, Digital Supply-Chain Management answers these questions.

This is what Actual Experience does... enabling business leaders to manage their digital investment better by providing the linkage between IT and business KPIs.

AE's breakthrough in Digital Supply-Chain Management has been recognised with two awards for innovation:

    the 2012 IET Innovation Award for Information Technology 
    the 2013 BCS UK IT Industry Innovation and Entrepreneurship Award.

The company has a growing portfolio of customers (see www.actual-experience.com/customers ) and partners (see www.actual-experience.com/company/partners ) and has completed a £4m funding round to support further expansion and internationalisation (see www.actual-experience.com/blog/?p=1156 ).

In addition, AE has been contracted by Ofcom (the UK communications regulator) to investigate the state of ‘Digital Britain’ and help to improve people’s experience of using the Internet. They are recruiting volunteers to run their free analytics software. Users will see the same analytics as Ofcom, live in their home – enabling them to see how well their broadband can support the nation’s most popular applications. They will also be able to compare their experience against a national standard for broadband excellence. This data, from hundreds of users, helps Actual Experience, Ofcom and broadband providers deliver better service for people all across the UK.

For more information and to get involved, sign-up at www.actual-experience.com/bbfix

The near-universal language of the Internet Protocol (IP) is the platform that brings together diverse applications and services, communicating over copper, fibre and wireless media.

Despite a diversity of uses that were never envisaged in the early days of the internet – from voice over IP (VoIP) to storing documents in the cloud – IP still works as a universal standard. Although this brings significant operational and cost benefits, there are substantial challenges too: many, varied types of traffic share the same infrastructure and produce such complex behaviours that it is difficult to predict the quality experienced by people using such a wide range of applications.

Blue-sky research in QM’s Electronic Engineering (now EECS) based Networks group over the past twenty years has specialised in the analysis and measurement of complex traffic and network behaviours in packet-based networking technologies. Pitts, Schormans, Mondragon and Phillips have addressed these challenges through four strands of research:

    Complexity modelling of network traffic and topology;
    Parsimonious queueing theory for packet-based network scenarios;
    Experimental design for measurement in non-linear and complex networks scenarios;
    Development of Quality of Experience (QoE) as a language for performance evaluation.

Mondragon, Pitts and Arrowsmith (Non-linear dynamics, Mathematical Sciences, QM) began working together in 1996, supported by a series of six EPSRC projects. They developed models of highly variable traffic behaviour (R2) and topology growth (R4) that explain the complexities of large-scale packet-based networks. This led to joint papers (R3) and three patents with Lucent in measuring complexity in 4G mobile networks, and CASE awards with BT labs in internet traffic modelling. The rich club concept pioneered by Mondragon (R4, R6) has been applied in disciplines ranging from international trade to neuroscience.

From 2002-9, with support from EPSRC, Nortel Networks, and the Royal Society, Phillips and Pitts focused on resource management techniques in fibre optic networks to take advantage of traffic variability by better scheduling traffic using uncertainty modelling (R5). This led to a joint patent with Nortel.

From 1995 to 2006, Pitts and Schormans built simple but powerful models that could be used in real time to model packet traffic (R1). The collaboration spawned two key insights: one (joint with Timotijevic, Vodafone) on the limits of measurability in networks, quantifying for the first time the non-linear relationship between accuracy and traffic; and one on the configuration of Quality of Service (QoS) mechanisms for graceful QoE degradation, which resulted in support in kind from Cisco for a study of the impact of QoS on network total cost of ownership.

The work on measurability led to Schormans and Pitts’ collaboration with Gilmour (Statistics Sciences Research Institute, Univ. of Southampton) and Moore (Computer Laboratory, Univ. of Cambridge) on the Theory of the Design of Experiments applied to optimal measurement of networks (2008 - present). The QoS work led to Pitts' development of QoE formulations as a language for performance evaluation and impairment diagnosis, research which has not been published for commercial reasons.

AE was formed in 2009 to offer these QoE formulations as a measurement and diagnosis service for businesses and consumers. Ongoing research funded by AE has led to the development of measures that reflect real human experience. This makes it possible to locate the infrastructure components in the digital supply chain that cause PQ impairments.

The key staff who carried out the research were Prof. Jonathan Pitts, Dr John Schormans, Dr Raul Mondragon, and Dr Chris Phillips.

    R1: Pitts, JM., Schormans, JA., Introduction to IP and ATM Design and Performance, 2000, Wiley.
    R2: Woolf M., Arrowsmith DK., Mondragón RJ., Pitts JM., Optimization and phase transitions in a chaotic model of data traffic, Phys. Rev. E, Volume 66, Issue 4, 2002.
    R3: Ho LTW., Samuel LG., Pitts JM., Applying emergent self-organising behaviour for the coordination of 4G networks using complexity metrics, Bell Labs Technical Journal 8(1), 2003.
    R4: Shi Zhou, Mondragon RJ., The rich-club phenomenon in the Internet topology, IEEE Communications Letters, vol.8, no.3, pp. 180-182, March 2004.
    R5: Dong S., Phillips C., and Friskney R., Differentiated-Resilience Provisioning in the Wavelength routed Optical Network, IEEE Journal of Lightwave Technology, pp. 667 - 673, Vol. 24 No. 2, February 2006. Volume 24, Issue 2.
    R6: Mondragon RJ., Topological Modelling of Large Networks, Philosophical Transactions A, Royal Society, London, doi:10/1098/rsta.2008.

dashboard_thumb

Back to top