User Experience and Device Monitoring Project

 

• The question 

Getting reliable indicators of the user experience of using applications has been a challenge since the use of IT has become widespread.

On the other hand, with the growth of cloud services and the virtualization of the workplace, there are more and more factors that affect the end user experience, and therefore, it is increasingly difficult to obtain reliable indicators of whether the interaction is appropriate or not.

Currently, different metrics are used, ranging from direct user surveys to obtaining indicators based on technical parameters that are assumed to positively or negatively affect the user experience.

Surveys have the drawback that we depend on an action on the part of the user and are therefore subjective, while indicators based on metrics fail to cover all possible factors and therefore end up generating many false negatives (the indicators show “ green” but the user complains).

With this project we propose to add an indicator based on the user's own interaction with the application, so that it does not require anything more from the user than his normal interaction with the applications (thus being an objective measure) and, on the other hand, to have identified the symptoms effectively to focus on finding the root causes.

Technological objectives:

A) Develop user behavior patterns in the use of different types of applications.

• Development of behavioral patterns at the level of use and movement of keyboard/touch screen and mouse (interaction time series), for different types of applications (office software, email managers, CAD and engineering tools, web browsing), and relating them to user satisfaction/frustration indices.

B) Develop calculation algorithms that parameterize the user experience, for different types of applications, based on obtaining direct data on user behavior.

• Development of calculation algorithms to obtain user experience indices, integrating behavior pattern data, and integrating infrastructure and communications data, allowing a composite indicator of user experience to be obtained.

• Development of an efficiency indicator, which based on the calculation algorithm allows the user to be classified into type groups (such as nervous, irrational, rational, etc.)

C) Develop a specific “agent” to collect real-time data on keyboard and mouse usage, minimizing the need for bandwidth and CPU computing power.

• Development of an agent for obtaining interaction data (Keylogger) based on remote visualization protocols, which allows capturing interaction series, to later be processed and interpreted by the calculation algorithm of technological objective B.

• The analysis

For this purpose, a large amount of data is collected from the devices at a very high frequency, which must be stored, and which is used for training machine learning models and developing indicators.

 The technologies used for the models are Python libraries such as Pytorch or Scikit-Learn. Models are deployed in production in Azure with continous deployment using APIs. For realtime RabbitMQ and Redis are used to store models and profiles.  Data are stored in ElasticSearch.

We can see here an index used in the algorithm:

Flow chart of the analysis:

Platform DeterminedAI used for training:

• The result

The result is Overa Activity:

It contains severals screens that allow customers to view any degradation in real time, whether of the devices themselves or the work of their employees.