Publication Year
Article Type

On Aspects of Quality Assessment Criteria of Mobile Banking Applications in Poland

Empirical study

Citation Download PDF

International Journal of Innovation and Economic Development
Volume 5, Issue 6, February 2020, Pages 55-70

On Aspects of Quality Assessment Criteria of Mobile Banking Applications in Poland

DOI: 10.18775/ijied.1849-7551-7020.2015.56.2005
URL: http://dx.doi.org/10.18775/ijied.1849-7551-7020.2015.56.2005

1 Witold Chmielarz, 2 Konrad Łuczak, 3 Marek Zborowski

1,3 Faculty of Management, University of Warsaw, Warsaw, Poland
2Digital Business Professional, Partner in KL Consulting Ltd., Warsaw, Poland

Abstract: The main objective of this article is to present the findings and to analyse a survey questionnaire used to assess the quality of mobile banking applications offered by commercial banks in Poland. The study carried out by the authors concerned the importance of the evaluation criteria adopted in the studies into the quality of mobile applications for individual users. The findings discussed in the article focus on mobile banking applications offered by universal banks in Poland which are available for mobile devices running on the Android, iOS and Windows operating systems. The structure of the article consists of presenting the general assumptions of the study, describing the methodology and the research sample, analysing the obtained findings as well as relevant discussions and interpretations. The paper is of a quantitative nature and has been conducted on a selected sample of respondents using banking services and products. The presented study and its analysis will empirically verify how individual clients assess the quality of banking applications and which criteria of this evaluation are most important to them. The authors’ original contribution was: specifying the criteria used for websites’ evaluation as the main indicators of the perception of the quality of websites; identifying the best e-banking websites and formulating conclusions which may constitute the starting point for designing an efficient system for quality management of e-services in the sectors.

Keywords: Importance of quality criteria, mobile banking services and products, mobile banking websites quality, website quality assessment

On Aspects of Quality Assessment Criteria of Mobile Banking Applications in Poland

1. Introduction

The most frequently used and the most intuitive classification of electronic banking is the division according to the communication channels used by the clients to contact their bank (Chmielarz, 2005). The most important and the most dynamically developing type of banking at present is mobile banking, which owing to the technological capabilities which emerged in recent years has gained a new and unprecedented dimension. An important element of this development is a commercial success and greater popularity of smartphones. Producers of these devices forecast that in 2019 in Europe the number of smartphones will reach nearly 760 million, and these devices will account for over 80% of all mobile phones globally (Eurostat, 2017; Ułan, 2017). As a consequence of the mass scale popularity of these devices in the last few years, there emerged a large market for mobile applications which virtually replaced the technologies such as WAP, SMS or USSD which were previously functioning in this channel. Thus, mobile banking is developing extremely fast, and clients are eager to use the opportunity to contact their banks via a smartphone. Using the knowledge resulting from innovation and drawing on the psychological determinants in consumer behaviour form the basis for building a closer relation between the banks and their clients using the software running on mobile phones or tablets. The applications designed for clients’ personal devices constitute the next milestone in the development of banking systems creating virtually unlimited possibilities of personalising the offer and establishing a closer relation and more direct contact with individual clients.

The research and the above estimates indicate that using mobile devices, including also a direct relation between the consumer and his or her bank, impacts everyday life of a modern man. It is important to note that this phenomenon is considered to be one of the most important sociological phenomena in recent years (King, 2013). A mobile phone is perceived as a personal and personalised device, and therefore all forms of information conveyed using this device constitute the most private kind of communication. The analysis and assessment of mobile applications, as a tool used to exchange information between the client and the bank, is a very interesting field of research in a broad cognitive context.

The authors hope that the presented study and its analysis will empirically verify how individual clients assess the quality of banking applications and which criteria of this evaluation are most important to them.

2. Assumptions Of The Research And Methodology Of The Survey Questionnaire

The main objective of the survey was to assess the quality of mobile applications offered by retail banks in Poland and to indicate the relevance of particular assessment criteria from the users’ point of view. The study into the quality of mobile applications requires selecting an appropriate set of quality assessment criteria (Nielsen, 2002). The set may be established based on the analyses of other groups of factors applied in the evaluations of similar IT solutions in different sectors (Wang, Bovik Sheikh, Simincelli, 2004). The selection of the group of relevant factors should be preceded with the analysis of relevant scientific literature on the quality of IT systems, which helps to establish many common features indicated in the rankings carried out in different IT fields (Moustakis, Litos, Dalivigas, Tsironis; 2011. Regarding IT systems, including computer software, the definition of the quality according to an international ISO 9126 norm relates to all the features and properties of software which affect the programs’ and applications’ ability to satisfy various specific or implied needs (ISO/IEC IS 9126, 1991). In line with this definition, the features which impact the quality of software may be divided into two areas: the interface and the functioning of the application. The former aspect is responsible for a broadly defined comfort of users’ use of the software, the latter for an equally widely understood proper functioning of the software (Iwaarden,  Wiele, Ball, Millen, 2004). The multitude of definitions and views concerning the quality of computer software points to the diversity of opinions we encounter when experts, designers and users discuss this particular phenomenon (Grigoroudis, Litos, Moustakis, Politis, Tsironis, 2008). This article examines the quality from the perspective of an application regarded as an important element of the bank’s IT system (Chmielarz, Szumski, Zborowski, 2011; Beck, 2003). Thus, its quality will be understood as the ability of the application to satisfy clients’ expectations and needs while meeting specific requirements resulting from the regulatory framework of the financial market and the specificity of the mobile channel (Virpi, Kaikkonen, 2003; Athanassopoulus, 2000). In this paper, the authors will not analyse the question of the quality related to the banks’ operations or marketing activities and related costs of devising and making the applications available in the market.

In order to assess the quality of banking applications, the presented study applied the criteria which have been discussed in detail by the authors in their earlier publications (Zborowski, Łuczak, 2016). The examined criteria include 24 features classified in 7 groups referred to as the main criteria, see table 1.

Table 1: Description of the criteria used to assess the quality of banking websites

Criteria and their features Description
A.  Perception of a mobile application Determines the user’s approach and his/her attitude towards the use of the application and is closely related to technology acceptance model (TAM) and the TTF theory describing the technology and the compliance with the purpose of using it. The perception is also associated with the assumptions of creating application interfaces in accordance with the UCD user-oriented design principles as well as the design based on UXD user experience.
User-friendlinessUsing the application is easy, the user does not feel irritated or tired while using it.
IntuitivenessUsing the application is intuitive, consistent with the user’s expectations and experience. It is relatively easy for the client to learn how to use it.
StabilityThe application works quickly and in a stable way, it does not crash.
B.  Navigation It results from the concept of logical design and the design based on the user experience (UX). It refers to the user’s impressions without indicating a specific element of the design or the functionality of the application.
Navigating the applicationNavigating the application is easy, and the navigation bar and the bar with command icons help to manage the content of the application.
Navigation schemeThe structure of the application, the way of presenting subsequent levels, e.g. sequential scheme (subsequent screens are arranged in order) or a hierarchical scheme (in the form of a tree).
Layout of elementsThe number and the arrangement of elements on the application screen is clear and readable.
GesturesThe application uses the functionalities of a touch screen, e.g. to turn pages (backward and forward), “pinch”, “touch and hold” to select an item.
System (physical and virtual) buttonsApplication uses the telephone button systems, e.g. Back, Home, Recently used applications.
C.  Menu layout The features and hierarchy of products and services for the information and transactional application, i.e. providing the user with information and enabling him or her to carry out banking operations, constitute separate criterion.
The order of the menu itemsThe order of the items (e.g. products, information) in the menu is correct and intuitive; the items are logically grouped.
Layout (block system)Presentation of the content: linear (moved down or sideways) or tiled (dividing content into several screens).
Correctness of indicationsThe names of items in the menu are unambiguous and precise. They indicate the next steps, according to the user’s expectations.
D.  Functionalities They result from the assumptions of logical design as well as the theory of adequacy of tasks for TTF technology. The components of this criterion are the application element that allows you to find the proper information and carry out specific tasks.
SearchThe search field (e.g. of a product or information) is accessible, and the search is accurate and precise.
PersonalisationThe ability to customise the appearance of the application to meet the user’s needs (e.g. background, layout, the arrangement of items) and forms of presenting information and reports.
ExportingThe possibility to generate files containing e.g. transaction history and sending it to the designated email address.
Recommending to a friendThe possibility to recommend an application or a banking product via the application in social media (e.g. Facebook).
HelpAids, additional materials and terms of use are easily accessible and understandable. The application offers tutorials, and the user is redirected to a presentation, tutorial or text mode.
E.  Visualisation An element of logical application and design oriented towards user’s experience UXD. This element concerns the sense of aesthetics which is a subjective view of each user. The feeling has to be positive and consistent with the rules of visualisation adopted by the bank.
Graphic elementsVisual elements (logo, graphics, and animation) are in line with the character of the application.
Colour schemeThe selection of colours used matches the users’ expectations and the bank’s image.
F.   TextAn element the corresponding to the clarity of the information which is crucial for the quality of customer service. The information and operational nature of the application requires users’ focusing on the presented content as well as on the interactivity when entering the data.
Text clarityTexts in the application are concise, precise and understandable.
Text styleFont selection (size, colour), punctuation which makes it easier for the client to use the application.
Entering textTyping on a virtual keyword in the application. The size of buttons is right and the appropriate alpha- and numeric keyboard is used.
G.  Security The criterion of a special character, when the access terminal to bank systems uses a personal client’s device which is connected to the Internet.
Authentication and authorisationThe uniqueness of user ID, the length of the password and PIN ensure secure banking operations. Only an authorised person can use the resources of the application.
Data integrityThe application ensures that the data and information in the account have not been changed or deleted by unauthorised persons.
ConfidentialityThe confidentiality of data and information concerning the user of the application, including the protection of his/her personal data is guaranteed.


The applications running on Android, iOS and Windows operating systems were the object of the study. The authors applied the research procedure consisting of the following stages: the selection and justification of the research sample, the construction a survey on the assessment of the quality of banking applications, the use of the scoring method for comparative analysis, analysis and subsequent discussion of the findings.

The study has been carried out with the application of the CAWI (Computer Associated Web Interview) method in the period of 27 February – 18 March 2015 among the members of the epanel.pl research panel. The research tool which was applied in the present research was a standardised, electronic survey questionnaire which was circulated online. The selection of the respondents was a case of a random-purposive sampling. The respondents were individuals meeting the following criteria: they held a bank account, owned a mobile device running on one of the three operating systems: Android, iOS or Windows, and they used m-banking applications and the Internet.

15,933 respondents were invited to participate in the study. The authors collected 1,525 fully completed surveys. A closer analysis of the data for the selected sample pointed to some deviations concerning 54 respondents who did not use m-banking applications for mobile devices during the research. Therefore, to conduct further analyses required for the evaluation, the authors carried out specific adjustments regarding the data, which ensured the reliability of the assessment of the considered mobile banking applications. As a result, the final research sample obtained in this way included 1,471questionnaires.

The findings related to particular questions of the survey questionnaire were illustrated with the graphs where the number of responses (N) to particular questions is presented together with the information concerning the completion of the questionnaire – i.e. selecting a single response or multiple responses. In the case of choosing one answer, the values add up to 100% with the assumption of rounding errors.

3. Description of the Research Sample and Analysis of the Obtained Findings

3.1 Problem Statement

Among the respondents, there were 53% of women and 47% of men, and the sample included mainly young individuals – more than half, i.e. 56% were aged 25-34. Every fifth adult was under 24, and the respondents aged 35-44 accounted for 17% of the sample. 6% share were individuals over 45. Among the respondents, the majority of individuals declared having completed higher (48%) or secondary (29%) level education institutions. Every fifth respondent declared holding a Bachelor’s degree or indicated incomplete higher education. The responses related to primary level education and vocational schools were selected by 2% and 1% respectively. Over half of the respondents (51%) lived in large cities with more than 200,000 inhabitants, one third lived in medium-sized cities with 21,000-200,000, and every tenth survey participant lived in a village. The majority were employed on the basis of employment contracts (64%) or were students (14%). 8% of the respondents worked on the basis of the contract of mandate, and 6% were self-employed. The number of unemployed individuals was the same as those taking care of their households or raising children, i.e. 3% each. Pensioners and retirees accounted for less than 2% of the sample.

3.2 Analysis of the Findings Regarding the Evaluation of the Application Quality

The research covered 16 mobile applications offered by the following banks: Alior Bank, Bank BPH, Bank Zachodni WBK, BNP Paribas, Citi Handlowy, Eurobank, Getin Noble Bank, ING Bank Śląski, Inteligo, mBank, Meritum Bank, Millennium Bank, Orange Finanse, Pekao S.A., PKO BP, Raiffeisen Polbank and T-Mobile Usługi Bankowe. The evaluation of the quality of applications did not include Bank Spółdzielczy PBS Bank and Plus Bank because the authors received only one survey questionnaire concerning the evaluation of the respective applications.

As a result of the conducted quantitative study, the authors obtained the results which are presented in the tables below. The respondents were asked to provide the responses to the following questions: „What is your assessment of the bank application [variable: name of the bank the respondent uses] in terms of the features such as [variable: name of the feature within a given criterion with a brief description]?”

The respondent evaluated each feature using a scale from 1 – Very bad to 5 – Very good.

Table 2 contains the basic characteristics of the application evaluations made by the respondents. The average value of the application quality assessments for all features in the examined sample amounted to 3,966, and the value of the dominant for the entire sample was 4. The average deviation from the average was 0.804, which resulted in the value of the coefficient of variation reaching the level of 20%. Although the overall score of the quality assessment was positive, it still leaves much to be desired.

The overall assessment of the quality of banking applications proposed by the authors consists in the measurement of quality criteria classified in the main criteria sets. The Alfa Cronbach coefficient for the selected measurement scale, where the 1st position was the lowest score, and 5 was the highest score for 24 features, amounted to 0.958. This points to the high degree of reliability of the research, its repeatability as well as the independence from the researcher or other study-related circumstances. The basic statistical characteristics obtained for particular features, i.e. average, deviation, the correlation of the feature with the overall assessment of the quality of the application are presented in Table 4. It is important to point to the high variability of assessments, which is indicated by the high value of the coefficient of variation falling in the range of 17,5%-24.8%

Table 2: Basic characteristics of overall quality assessments of mobile banking applications made by users

Descriptive statistics for the examined sampleValue
Average value3.966
Standard deviation0.804
Coefficient of variation20.3%

A markedly greater variation is observed in the case of criteria and features which receive the lowest scores, e.g. Functionality and Navigation. This tendency is characteristic for the assessments below the average, where there occurs the so-called dispersed dissatisfaction stemming from the diversity of needs and requirements of the users occurring together with the low level of personalisation of the application functionality. This, in turn, results from different weights attached to particular quality criteria. In the case of the features and criteria ranked above the average, the consistency of assessments is relatively higher, because the values of the coefficient of variation are lower. These ratings are less dispersed than in the case of the average quality assessment, for which the average coefficient of variation amounts to 20.3% (Table 2).

For all 24 features considered in the study, the authors have carried out an analysis of the correlation which describes the relation between the variables. All indicators related to Pearson correlation between the overall quality assessment and the scores assigned for individual features are statistically significant (α<0.05). They confirm a strong positive correlation between the examined features and the assessment of application quality.

The weakest relation with the general assessment of the application quality was indicated in the case of a feature which obtains the lowest scores, i.e. recommending to a friend (r=0.616). Comparing the averages for the features with the correlation coefficient, one may observe that the assessments are in line with the overall assessment of the quality of the application. The group of features which received the scores ranging above the average, where we observe a weak relation with an overall assessment of the quality, included the features such as: personalization (r=0.642), exporting (r=0.65), or gestures (r=0.656). The features such as search and help received scores below the average and they have an average impact on the overall quality assessment.

The features which have a strong influence on the overall assessment of the application quality, and simultaneously were rated below the average are the navigating scheme (  =3.96, r=0.78) and system buttons  (  =3.93, r=0.749). Improving these particular aspects will significantly affect the overall assessment of the quality of the application.

Table 3: Correlation coefficients for the assessment of particular features with the overall quality assessment

FeatureAverageDeviationCorrelation coefficient (r)
Recommendation to a friend3.570.870.616
Authentication and authorisation4.060.740.68
Entering text4.060.780.688
Colour scheme4.160.770.708
Data integrity4.030.700.714
Graphic elements4.140.750.732
Text style4.150.710.744
System buttons3.930.780.749
Block layout4.050.750.755
Text clarity4.190.670.758
Correctness of the indications4.100.710.772
Orders of the items4.050.730.776
Navigation scheme3.960.760.78
Application navigation4.080.750.781
Elements distribution4.060.740.791

The analysis of the data with the application of a simple scoring method indicated that the respondents assigned relatively good scores for particular features of the examined applications. However, there were also those which received average scores. The best average scores were obtained in the case of the Visualisation criterion, followed by the criteria pertaining to Text and Perception of the application. The lowest average value in the study was assigned in the case of the Functionality criterion, slightly worse than the Navigation criterion. Table 4 presents the arithmetic mean of the evaluations of the features in the group of a particular criterion on the basis of a scoring method as well as basic statistical measures for particular criteria.

Table 4: Average rating of the features in the group of criteria, using a scoring method

Bank/ CriterionAverage of the assessments of features in the criteria set, on the basis of a scoring method
Perception of the applicationNavigationMenu constructionFunctionalityVisualisationTextSecurity
Dominant within the criterion4.0004.0004.0004.0004.0004.0004.000
Average for a criterion4.1123.8554.0613.6244.1464.1274.044
Standard deviation0.7560.7550.7290.8990.7600.7230.711
Variability coefficient0.1840.1960.1800.2480.1830.1750.176
Preference vector for criteria0.1470.1380.1450.1300.1480.1480.145

Analysing the period of time of clients’ using the application, it can be observed that the average assessment of the applications’ quality shows an increasing tendency in line with the extension of the period of using the tool, see Figure 1. The highest scores in the quality assessment could be found in the group of regular clients using the application for over 4 years. This suggests that together with the extension of the period of cooperation with the bank, the clients were more satisfied with the quality of the services provided via this channel. Simultaneously, it may be concluded that together with the acquisition of skills related to the use of the application, users’ expectations tend to decrease, both regarding the application and the bank itself.

Interesting conclusions can also be drawn on the basis of a similar analysis carried out for particular applications analysed in the study, see Figure 2. In the case of two applications most commonly used in the examined study sample, i.e. mBank and ING Bank Śląski, the average quality assessments are the highest in the group of clients who use the application for more than half a year and for a period shorter than two years. Subsequently, the ratings drop to increase once again in the group of regular clients using the application for over four years. In the case of the application of Bank Zachodni WBK, the average assessments fell rapidly when moving from a group of new clients (up to 6 months) to a group of medium-term clients (up to 2 years), to rise rapidly and decrease once again within the group of individuals using the application for the longest period of time.

In the case of two of the biggest retail banks, i.e. PKO BP and Pekao the average scores of the quality of the application are very high in a group of new clients, then they are rapidly falling and rising again in the group of relatively new clients (up to 2 years). At the same time, the scores of Pekao application reached the highest average in the group of medium-term clients using the application from 2 to 4 years (  =4.10). The assessments of the PKO BP application are increasing in the group of clients using the application for more than 4 years to reach the average of   =4,03. There are significant differences in the average scores received for the quality in different segments of users which may point to the discrepancies in the clients’ expectations at different stages of the cooperation with the bank. Interestingly, nearly half of the respondents participating in the study were categorised in two groups, i.e. from 7 to 12 months and from 1 year to 2 years where the disruption of the increasing tendency of quality assessments is visible.

Taking into account the frequency of using the application where 90% of the survey participants used mobile banking at least a few times a month, and where 20% use it at least once per month, it may be concluded that experienced users are willing to use new and more advanced financial services via mobile applications. Therefore, their opinions are particularly important from the point of view of devising a model of the application and ensuring proper communication of the clients and the bank.

The evaluation of the application quality according to individual features was carried out with the assumption that a respondent assesses every considered element on a scale from 1 – Very bad to 5 – Very good. Additionally, there was a rotation of the sets of criteria, and within each criterion the features were also rotated. The analysis of specific data in particular main criteria sets shows that, when evaluating the Functionality criterion, the respondents assigned worst scores for the features such as: exporting and recommending to a friend. These attributes were the only ones in the study that had the predominant number of average scores (3 – neither good nor bad). The feature of recommending to a friend was assessed as average or bad in as many as 11 applications out of the 16 options analysed in the study and exporting – in 7.

The structure of the quality assessment of particular features in the Functionality criterion is shown in Figure 1. It indicates that over 10% of the respondents expressed the opinions: Rather bad or Very bad with regard to exporting and personalisation of the application. This was the worst result recorded in the study.

Figure 1. Structure of the assessment of features in Functionality criterion

Source: The authors’ own work based on the conducted research, sample N=1471

The second criterion which recorded the lowest scores was Navigation and the features which received the least points were gestures and system buttons. Even though in the case of these evaluations the average assessment was less than good, still the responses: Rather good were dominating for all the banks included in the research. The best scores in the Navigation criterion were assigned for the feature of navigating the application, where the percentage of good and very good scores reached over 83%, Figure 2.

Figure 2. Structure of the assessments of features in the Menu construction criterion

Source: The authors’ own work based on the conducted research, sample N=1471

The respondents assigned good scores for the Menu construction and Security, the results of which are presented in Tables 8 and 9. The average value of the assessments of particular features for these criteria was always above good, and the most frequently selected response was Rather good.

In the case of Menu construction criterion, the highest score was obtained for the feature of the correctness of indications, and the lowest score for the order of menu items, see Figure 3. However, in each case, the features obtained over 80% of good and very good scores.

Figure 3. Structure of the assessments of features in the Menu construction criterion

Source: The authors’ own work based on the conducted research, sample N=1471

The features such as authentication, authorisation and confidentiality received high scores within the Security criterion. Slightly lower scores were assigned for data integrity where the structure of the evaluation is still at a very high level, see Figure 4.

Figure 4. Structure of the assessments of features in the Security criterion

Source: The authors’ own work based on the conducted research, sample N=1471

Among the highest rated main criteria, the best scores were recorded in the case of text clarity in Text criterion. Simultaneously, it is the highest score in the study among all the assessments of the features. In the case of this feature, less than 10% of the survey participants assigned the scores which were lower than good, see Figure 5. Even though in the case of this criterion, the lowest scores were assigned for entering the text, still the score was better than the average value calculated for all features.

Figure 5. Structure of the assessments of features in the Text criterion

Source: The authors’ own work based on the conducted research, N=1471

High scores were assigned to user-friendliness in the Perception criterion, and subsequently, with equally high scores: intuitiveness and stability of the application. Only 3-4% of the sample assessed these features as Bad or Very bad, see Figure 6.

Figure 6. Structure of the assessments of features in the Perception criterion

Source: The authors’ own work based on the conducted research, N=1471

Also, high scores were recorded in the case of Visualisation. The feature which obtained the highest scores was the colour scheme. The colour scheme was the feature which obtained the greatest number of very good scores in the study. It is important to note that graphics received equally high scores, see Figure 7.

Figure 7. Structure of the assessments of features in the Visualisation criterion

Source: The authors’ own work based on the conducted research, sample N=1471

The research findings constitute the basis for detailed considerations concerning the assessment of the quality of m-banking applications. These analyses should be carried out following the verification of several measurement models. One of them consists in lack of differentiation between the relevance of particular criteria and features in assessing the quality of the application based on the scoring criterion presented above. The analysis conducted with the application of a scoring method is frequently criticised for its subjectivity, as it does not consider particular measurements which are characteristic for multidimensional analysis methods. However, in this case, the size of the research sample N = 1 471 allows the results to be considered as representative in the case of users of the m-banking application.

3.3 The Declared Relevance of the Criteria Used for the Assessment of the Application Quality

An important element of the presented quantitative research was users’ establishing of the hierarchy of the relevance of the criteria indicated by the author. The respondents taking part in the quantitative study were to evaluate how important particular criteria are in the context of the application which they use. The question was as follows: “How important, in your view, are the particular criteria of the bank’s [variable: bank name] mobile application?” The scope of possible responses ranged from 1 – Definitely irrelevant to 5 – Definitely relevant.

Figure 8 presents the summary of the assessments of the relevance of individual criteria. According to the respondents, the most important criterion in terms of the quality of the m-banking application is Security which includes the features such as authentication and authorisation, data integrity and confidentiality. In this case, we record nearly twice as many Definitely relevant responses than in the case of the subsequently presented research criteria. The relevance of the Security criterion is in line with the findings of other studies focusing on the electronic channels of the distribution of banking products and services, e.g. the Internet, where the issue of guaranteed sufficient security is listed as the most relevant aspect among the factors stimulating the development of banking (Polasik, 2012). Therefore, one of the most important activities should be to provide users with relevant security measures through applying technological solutions, e.g. tokens used to authorise transactions, to educate clients on how to engage in secure transactions and overcome the fear, anxiety or lack of trust in modern technologies.

The subsequent places were occupied by a group of criteria such as Navigating, Perception of the application, Text and Menu construction which obtained largely the same scores (83-85%) regarding the relevance of these criteria in the quality evaluation. Proper implementation of the features grouped in these criteria points to the fact that in the case of the information and transactional applications, the aspect of general attractiveness of the design and ease of navigating the screens appear to be just as important as the clarity of the displayed content.

The expanded service and product range available in the application focuses on the importance of a clear menu and an efficient navigating system. However, it appears that the Functionality of the application is nearly just as important. For many respondents, this category was assessed as Definitely relevant and has obtained a similar score (i.e. 33%) as in the above-mentioned group of categories. However, in the evaluations there was also a marked number of responses indicating its irrelevance or indifference to it, amounting to over 20%. Most likely, the latter may be related to the reluctance with regard to using these elements, no skills or bad experience when using the application. The functionalities corresponding to the most common reasons for using the application include: a well-designed login area, the ability to personalise the interface to facilitate the use of the application, the search engine which is efficient and helps to find answers to the queries, the assistance provided by employees or available in the form of electronic tutorials and demo versions.

An even greater percentage of the responses indicating the irrelevance of specific factors was indicated in the case of Visualisation criterion. Every third respondent gave such an answer, hence this feature was ranked last in the research into the relevance of the criteria applied in the quality evaluation of the application. In practice, although the graphics provide users with a clear and readable layout, consisting in the application of large icons and navigation buttons, the users are aware that in most cases, they cannot adapt the interface to suit their individual needs. The presentation is determined by the bank’s branding policy which, apart from some exceptions, does not allow for changing the colours of the background of the application. No possibility to personalise or arrange the elements on the screen to e.g. distinguish the cards and scrolls with instructions may also result in the low assessment of this criterion.

Figure 8. The structure of the assessment of the application according to individual criteria

Source: The authors’ own work on the basis of the conducted research, sample N=1471.

4. Discussion of the Findings and Conclusions

Table 5 presents the comparison of the values of the arithmetic mean of the assessments of criteria relevance weights in the examined applications. Comparing the particular criteria assessments obtained based on the measurement model with their relevance, it is possible to create a matrix of the assessment parameters, see Figure 9.

Table 5. Comparison of the arithmetic mean value of the relevance weighs of the main criteria

No.CriterionAssessment of the criterionSurveyed relevance
5.Menu structure4.0614.10


Drawing the lines of the matrix into quadrants with the variance divided in half, the authors obtained four subsets of criteria.

The first subset consists of the following criteria: Security, Perception of the application and Text. These measurements turned out to be relatively important for the respondents, and they ranked above 4.129 (their assessments exceed the average of all evaluations which is 3.968). The application designers should focus on these criteria to ensure and sustain a high level of fulfilment of these criteria.

The second subset contains the Menu and Visualisation criteria. These aspects are not very important for the users even though they received relatively good scores. In the case of designing m-banking applications, these aspects may be seen as potential savings for the service providers. For example, the respondents assigned good scores for the applications’ colour schemes. However, they have no influence on this aspect since this particular element depends on the complex branding policy of the bank, which concerns all channels of communication with the client, not only those related to mobile banking.

The third area is the Functionality criterion, which is seen as one of limited relevance and the aspect which received relatively low scores. When designing an application, it is worthwhile to examine this particular factor which received the lowest scores,  =3.624. The average importance of this feature means that designers could improve the overall assessment of the application’s quality focusing on this particular issue. The analysis of the model used for the assessment of banks examined in the study indicates that the users evaluating this particular feature attach more importance to this criterion than to the Visualisation of the application.

Figure 9. The matrix of the relevance of quality assessment criteria

The fourth, last subset includes the Navigation criterion which is evaluated below the average, but its relevance for the survey participants is higher than the average. When designing an application, it is important to pay attention to the elements which influence this criterion since their improvement may bring the greatest change in the overall perception of the application’s quality. In terms of relevance, this criterion gives way only to Security, which clearly shows that the elements related to the navigation of the application are crucial for the evaluation of the quality of a mobile application.

Summing up, the distribution of the relevance of the criteria as qualitative attributes points to the elements which designers need to focus on when designing a banking application. The key aspects include the necessity to maintain high quality of the application in terms of security, user-friendliness, i.e. the clarity and comprehensibility of the texts and a greater focus on navigating the application. The features included in the last criterion are related to the speed and ease of finding the information in the application. From the point of view of the users, these aspects are more important than the functionalities which are available in the application. Thus, taking all the above into consideration, it is important to note that focusing on improving the visual aspects and elements of the menu alone will have little impact on the perception of the overall quality of the mobile banking application.


  • Athanassopoulos A. (2000). Customer Satisfaction Cues To Support Market Segmentation and Explain Switching Behavior, Journal of Business Research No 47, 191–207. New York: Elsevier Science Inc.
  • Beck, S. (2003). Evaluation Criteria: The Good, the Bad and the Ugly: or, why It’s Good Idea to Evaluate Web Sources. Retrieved 20.06.2018 from: http://lib.nmsu.edu/instruction/evalcrit.html.
  • Chmielarz W. (2005). Systemy elektronicznej bankowości. Warsaw:
  • Chmielarz W., Szumski O., Zborowski M. (2011). Kompleksowe metody ewaluacji jakości serwisów internetowych. Warsaw: Wydawnictwo Naukowe Wydziału Zarządzania Uniwersytetu Warszawskiego,
  • Number of smartphone users worldwide from 2014 to 2020 (in billions) (2017), Retrieved 8.03.2018 from https://www.statista.com/statistics/330695/number-of-smartphone-users-worldwide/.
  • Grigoroudis E., Litos Ch., Moustakis V., Politis Y., Tsironis L. (2008). The assessment of user-perceived web quality: Application of a satisfaction benchmarking approach, European Journal of Operational Research, No 187, pp.1346–1357. New York: Elsevier Science Inc.
  • ISO/IEC IS 9126 (1991). Information Technology – Software Product Evaluation – Quality Characteristics and Guide Lines for Their Use. Geneva: ISO.
  • Iwaarden, J., Wiele T., L., Ball L., Millen, R. (2004). Perceptions about the quality of web sites: A survey amongst students at Northeastern University and Erasmus University, Vol. 41 (8), pp. 947–959. Information and Management. Amsterdam: Elsevier Publishers B.V.
  • King B. (2013). Bank 3.0 – nowy wymiar bankowości. Warsaw: Studio Emka.
  • Moustakis V., Litos Ch., Dalivigas A., Tsironis L. (2011), Website Quality Assessment Criteria), Retrieved 21.05.2018 from: https://www.researchgate.net/publication/220918684_Website_Quality_Assessment_Criteria.
  • Nielsen, J. (2002), Designing Web Usability: The Practice of Simplicity. Indianapolis: New Riders Publishing.
  • Polasik M. (2012). Bankowość elektroniczna. Istota – stan – perspektywy. Warsaw: CeDeWu.
  • Ułan G. (2017). CBOS sprawdził, ilu Polaków korzysta jeszcze z klasycznych telefonów komórkowych. Retrieved 15.01.2018 from https://antyweb.pl/cbos-sprawdzil-ilu-polakow-korzysta-jeszcze-z-klasycznych-telefonow-komorkowych/.
  • Virpi, R., Kaikkonen, A., (2003). Acceptable download times in the mobile internet. In: Proceedings of the 10th International Conference on Human-Computer Interaction, No 4, pp. 1467–1472. New Jersey: Lawrence Erlbaum Associates.
  • Wang Z., Bovik A., Sheikh H., Simoncelli E., (2004). Image Quality Assessment: From Error Visibility to Structural Similarity, No 4 (13), pp. 1-13, IEEE Transactions on Image Processing.
  • Zborowski M., Łuczak K. (2016). Dobór kryteriów oceny bankowych aplikacji mobilnych. In W. Chmielarz (ed.). Mobilne aspekty technologii informacyjnych, pp. 161–186, Warsaw: Wydawnictwo Naukowe Wydziału Zarządzania Uniwersytetu Warszawskiego.

Comments are closed.

Subscribe To Newsletter
Be the first to get latest updates and exclusive content straight to your email inbox.
Stay Updated
Give it a try, you can unsubscribe anytime.