Evaluation of the Implementation of a Mobile Nursing Information System

Abstract

The implementation of a mobile nursing information system (MNIS) has been shown to be beneficial for nurses, taking away the burden of paperwork and enabling them to focus more on patient care. However, the factors that contribute to a successful MNIS implementation have been not been widely studied. This cross-sectional study collected data based on the DeLone & McLean Information System Success Model (1992) from 177 registered nurses in Taiwan to evaluate the implementation of a MNIS. Results showed that user satisfaction, managers’ support (β = .223, P = 0.001), system quality (β = 382, P = 0.001) and service quality (β = .161, P = 0.05) were significant predictor variables. The model consisting of the variables intention to use, managers’ support (β = .245, P < 0.001), years of RN experience (β = .093, P < 0.05), user satisfaction (β = .628, P < 0.001) produced an explained variance of 63.1%. An additional model consisting of system use and user satisfaction was also significant (β = .493, P < 0.001) and contributed to 24.3% of the total variance. Identified factors that affect the use of the MNIS can help hospital administration develop strategies for more successful implementation of MNIS.

Introduction

The latest advancements in mobile technology and wireless communication have led to the development of nursing information systems (NISs) in some healthcare organizations. Mobile nursing information systems (MNIS) have been implemented in mobile devices such as laptops for years; they are used as part of both an electronic health record and a health information system (Gephart, Carrington, & Finley, 2015). Nurses use MNIS in various work environments to respond quickly in articulating their information needs. The benefits include the reduction of duplicate documentation and automatic data processing (such as automatic visualized vital signs, nursing diagnoses accompanied by disease diagnosis, and ultimately patient care outcomes) (Lee, Sun, Kou, & Yeh, 2017). However, some negative results have been reported by nurses (Gephartet al., 2015). How to maximize the use of MNIS to optimize patient care and satisfy nurses’ increasing information needs is still a critical issue. Leveraging MNIS benefits requires further understanding of the practical issues facing the needs of an ever-changing society and the complexity of health care (Rouleau et al., 2017). This also requires knowledge on whether nursing leaders are able to take an active role in the implementation of MNIS to clinical practice.

The implementation of MNISs still has mixed results and some crucial challenges in some countries. An evaluation study of NISs reported that most nurses acknowledged NIS reduced documentation efforts (Sockolow, Rogers, Bowles, Hand, & George, 2014). But the researchers also identified several problems. First, most MNISs were mounted on nursing carts, which were usually placed in patient rooms or hallways; nurses were not comfortable that they had to stand several minutes during the use of MNIS cart. Second, MNISs are not as flexible as paper-based tools in sharing information during handover between nursing shifts (Sockolow et al., 2014), or in summarizing information on patient admission and discharge (Moghaddasi, Rabiei, Asadi, & Ostvan, 2017; Samadbeik, Shahrokhi, Saremian, Garavand, & Birjandi, 2017; Sockolow et al., 2014). Nurses’ dissatisfaction with NIS is often found because the system does not fully support nursing duties and workflow (Ahmadian, Dorosti, Khajouei, & Gohari, 2017; Moghaddasi et al., 2017; Valizadeh & Bagheriyeh, 2018; Vollmer, Prokosch, Evans, & Kuttler, 2016). These problems of inefficiency and dissatisfaction impact nurses’ willingness to use MNISs and may even hamper patient care.    

Influencing factors of end users’ perception are essential to successful system implementation. According to related technology acceptance models and system success theories, several key factors including organizational factors and task factors were shown to increase system use (Hsu & Wu, 2017; LaDage, Prasun, Linton, Kaiser, & Laskowski, 2015; Rashid, Abdulaziz, & Yasin, 2018; Valizadeh & Bagheriyeh, 2018; Zheng, Wang, Doll, Deng, & Williams, 2018). Others are individual willingness to use the technology and a fit between health IT and existing work practices (Vitari & Ologeanu-Taddei, 2018). These factors have not been comprehensively studied in the context of MNIS, which may lead to infrequent discussion in the literature. Moreover, previous studies on evaluation of MNISs is still limited. User acceptance of health information systems are continually studied (Shahzad, Jianqiu, Zia, Shaheen, & Sardar, 2018). User acceptance can be a prelude to or reflection of their satisfaction in using these systems, but acceptance is not representative of user satisfaction. A comprehensive method for gathering user satisfaction feedback is needed.

The Information System Success Model, known as the D&M IS Success Model, originated by DeLone and McLean in 1993, included dependent variables to measure the success and effectiveness of an information system. These variables are “information quality,” “system quality” and “service quality” (DeLone & McLean, 1993). The framework was refined in 2003 and is commonly used in evaluating the success of information systems (Delone & McLean, 2003). There are some missing elements of successfully implementing MNIS. Hsu & Wu (2017) explored the insufficient level of supportiveness from administration in health organizations, the system not being specifically designed for nursing workflows, and the lack of training for system operation. Health care managers or nursing executives are responsible for the successful deployment of MNIS. As a result, the model does not fully interpret the situation or context of MNIS.

Although NIS have been widely studied in the past, there are continual updates of the knowledge about information technologies with potential application in health care. A plethora of studies (but by far not all) point out that there are relationships between external variables of health information systems based upon user acceptance and technologically related models (Kisekka & Giboney, 2018). Lin (2017) applied an integrated model to predict nurses’ satisfaction with NIS by combining individual perception and technological functionality. But this research ignored organizational elements. Moreover, it is still unclear what the impact of administrative support is, whether positive or negative.

In the research context, this study assumed that a greater level of nursing leadership resulted in greater MNIS use. This study added elements into the success model to be more accurate and to address the effect of MNIS, including “managers’ support.” This research aims to explore more critical factors of the MNIS from the perspective of nurses.

Conceptual Framework

The adjusted D&M IS Success Model (Delone & McLean, 2003) was selected as the conceptual model for this study (Figure 1). This research suggests further investigation of the MNIS success model and defines the effectiveness of MNIS in hospitals. “Managers’ support” was added as an input factor, resulting in having these inputs: “managers’ support,” “information quality,” “system quality” and “service quality.” Demographic variables were also added. By applying the D&M IS Success Model to this study, the four inputs may affect the “intention to use” MNIS.  The “user satisfaction” may affect the “intention to use” of MNIS. Both “user satisfaction” and “intention to use” may affect the system use of MNIS.

Figure 1: Adjusted D&M IS Success Model

Systems quality measures technical success, referring to the extent to which systems are “user friendly” and can be used without any hassles. “System quality” is significantly related to “intention to use.” Information quality measures semantic success, referring to the system’s output value, as perceived by the respective user of the system. “Information quality” is significantly related to “intention to use.” Service quality measures the combination of service provided rather than the product itself. It is defined as users’ subjective assessment that the service they are receiving during systems testing and deployment is the service they expect. “Service quality” is significantly related with “intention to use.” Use defines whether the behavior is voluntary or mandatory, informed or uninformed, effective or ineffective. One of the major success dimensions of IS success is user satisfaction, which measures the users’ level of satisfaction with a system.

Furthermore, the relationship between the demographic information of staff and the factors in the model were analyzed to investigate whether the characteristics of staff have an effect on their opinions of MNIS.

Methods

Sampling and Setting

This research adopted a cross-sectional design based on purposive sampling techniques. The research was conducted at two Taiwanese hospitals, which adopted computerized nursing diagnoses in 1999 and an NIS in 2008. The target population consisted of nursing staff at each hospital. Out of the 10 nursing units that were using MNIS, eight were selected. Staff with the experience of using MNIS over three months were considered as potential candidates. The inclusion criteria were nursing staff who (a) were more than 20 years old, (b) worked more than three months and not part-time, (c) were able to read Chinese, and (d) were willing to participate in research and sign consent forms. Exclusion criteria were (a) an inability to answer questionnaires clearly; (b) working less than 3 months or employed part-time. A total of 225 registered nurses were invited to participate in the study.

The functionality of MNIS

The function of the MNIS included (1) recording nursing information, (2) nursing plan, (3) diagnosis related to nursing decision, (4) patient identification before procedure, (5) patient education, (6) attendance performance, and (7) official duties such as diet orders and adverse event reporting.

Measurement

The survey included demographic items and the IS Success Model scale. The demographic items included age, education, RN years, and RN level. The levels of registered nurse ranging from 0 to 4th level were assessed by performance appraisal in accordance with procedures agreed to by the Taiwan Nursing Association and the employer. The items of the constructs adapted from DeLone & McLean model consisted of seven dimensions, 43 items, graded by a five-point Likert scale that ranged from strongly agree (5) to strongly disagree (1). The validity and reliability of the tool were validated in previous research. The Cronbach’s alpha was reported to be 0.78 - 0.83 (Ahmadian, et al., 2017).

Process

The researchers introduced the study to each of the eight nursing units, including the purpose of the study, the method, the time required to complete the questionnaire, and how the data would be used after they were collected. A questionnaire survey was conducted at two hospitals in Taiwan that implement the same nursing information systems and devices. The participants filled out the questionnaire and returned it within five days.

Ethical Considerations

The study was conducted after the Institutional Review Board approval (reference number CGH- 98-0415B). Written informed consent forms were obtained from all participants. Participants were given instructions about the study purpose and the questionnaires. They were informed that participation in this study was voluntary, and that they could refuse or withdraw from the study at any point without giving any reason. Moreover, the participants were reassured that their responses would be kept confidential and their identities would not be revealed on research reports and publications of the study.

Data Analysis

The researchers used the IBM SPSS software to analyze the statistical results. Multiple regression analysis was used to examine model fit for each construct and its subconstructs (to assess the measurement model).

Results

The 225 nurses who fulfilled the inclusion criteria comprised the sample of the study, of which 216 completed the survey, 39 were excluded due to discontinued use of the MNIS, yielding a final response rate of 78.67% (N = 177). Table 1 details the demographic information of the respondents. The participants were all female, 83 (46.9%) of them were 26-30 years old, the average age was 28.57 (SD = 4.67) years, 96 (54.2%) had nursing experience for less than five years. Most of the participants had an associate (86, 48.6%) or bachelor’s degree (87, 49.2%), 78 (44.1%) were at the N2 level, meaning a registered nurse who has demonstrated competence in basic nursing practice and worked at least 2 years.

Table 1: Participant Demographics

Cronbach’s alpha for total scores was 0.889, and for constructs of MNIS success, the model ranged from 0.872 to 0.966 (Table 2). The descriptive statistics for these constructs indicated that most rated above the median score, managers’ support (mean = 28.76, SD = 5.24), system quality (mean = 21.09, SD = 5.20), information quality (mean = 13.61, SD = 2.96), service quality (mean = 9.18, SD = 2.7=56), intention to use (mean = 13.98, SD = 3.18), and user satisfaction (mean = 26.03, SD = 6.09).

Table 2: Cronbach’s alpha for total scores

Weak to strong correlations were found in the measures of the MNIS success (Table 3). There were positive correlations of quality-related constructs (system, r =.596, p < .01; information, r =.535, p < .01; service, r =.542, p < .01), and managers’ support (r =.712, p < .01) with intention to use. The quality related constructs also have positive correlations (system, r =.694, p < .01; information, r =.529, p < .01; service, r =.574, p < .01), and managers’ support (r =.588, p < .01) with user satisfaction. The correlation between system use and other constructs of MNIS were also weak to moderate.

Table 3: Correlations for all variables

Table 4 demonstrates the association of demographic and work characteristics with the constructs of the MNIS success model. As nurses’ experience were over 10 years, they were significantly more satisfied with the MINIS (F =4.988, P <.01). There were significant differences in “user satisfaction” and “intention to use” among four groups of RN level 0~4.

Table 4: Nursing characteristics and constructs of the MNIS success model

Multiple regression models were built to identify variables independently related to user satisfaction, intention to use and system Use (Table 5). First, the model of predicting for user satisfaction, managers’ support (β = .223, P < 0.001), system quality (β = 382, P < 0.001) and service quality (β = .161, P < 0.05) had a stronger impact on user emotion.

Table 5: Multiple regression results for each variable

The explained variance of user satisfaction model by each predictor is 56.5%. Second, the model of intention to use, managers’ support (β = .245, P < 0.001), years of RN experience (β = .093, P < 0.05), user satisfaction (β = .628, P < 0.001) was analyzed. The explained variance of intention to use model by each predictor is 63.1%. Third, the model of system use, user satisfaction was significant (β = .493, P < 0.001) and contributed to 24.3% of the total variance. The parameters of the MNIS success model were illustrated in Figure 2.

Figure 2: Parameters of the MNIS success model

Discussion

This study analyzed the factors of MNIS implementation according to the DeLone and McLean theory (1993, 2003) to provide an insight on how to successfully implement a mobile NIS by measuring users’ perceptions. The study found that the domain relationship of RN years and managers’ support for the use of the MNIS were positively related. The most powerful predictor about user satisfaction was system quality. Furthermore, user satisfaction is also the most powerful predictor in both behavioral intention and actual use of the MNIS. Managers’ support had a significant impact on the implementation of MNIS and nurses’ satisfaction. 

According to the result, most of the staff continued to use MNIS over three months and showed a positive attitude toward it. This shows that most of the staff were positive about the system. The characteristics of the staff, including age and service experiences, failed to show significant difference toward the factors of the model; this complies with past research that the attitudes of nursing staff would not be affected by personal factors such as age or service experience (Lin, 2017). A well-designed information system is desirable for nurses despite their individual differences (Gephart, et al., 2015).

It appears that the increase of intention to use MNIS leads to higher user satisfaction (Rashid, Abdulaziz, & Yasin, 2018). To increase the intention of using MNIS, an emphasis on manager support and system quality should be included. Previous research confirmed that manager support is crucial in adopting new technology, including providing enough training, clearly introducing the purpose and aim of the adoption, and keeping a close follow-up watch for negative effects of the adoption.

It is worth noticing that information quality does not necessarily affect the intention or the consistency of using MNIS. This may factor on whether MNIS could provide correct, comprehensive and timely information and whether the screen size fulfills the needs of work tasks but does not necessarily affect the staff using MNIS.

Limitations

Technology adoption affects the work of all the staff in the hospital. A limitation of this study is that it only recruits the opinions of nurses. Also, the data were collected from two medical institutes to ensure that the model does not merely represent a single phenomenon or setting. However, the two institutes were still established under the same foundation thus the generality of the model still needs to be validated. Moreover, this study only sampled eight units across these two hospitals. More data could be retrieved to reveal further insights into new technology adoption. Advanced MNIS design also requires more knowledge that can contribute to improving efficiency and satisfaction solutions in the era of healthcare system reform (Andargoli, Scheepers, Rajendran, & Sohal, 2017).

Conclusion

The results of this study established the variable “managers’ support” linking the use of MNIS, adding to our understanding of nursing management, which is critical in the context of MNIS implementation. Although DeLone and McLean’s theory (1993) provides a useful structured view to assess the successful factors of information systems, more nursing-specific elements should be considered for nurses as real end-users. Thus, a holistic viewpoint can better guide nursing informatics research.

The use of technology has been held as a symbol of a modern organization, and patients, nurses and health care managers can all benefit from the lessons learned during the evaluation of nursing information systems. This research considers the working environment of health organizations and proposed a model to facilitate the feasible evaluation of technology implementation in the future. There has been and will be in the future a need for updating MNIS to meet the changing requirements of the current health care system to serve both patients and nurses. A success model can help healthcare organizations determine specific information system dimensions and characteristics in need of improvement.

Citation: Cheng, C.C., Chan, C.L., Chen. L. & Guo, S.H.M. (Fall, 2019). Evaluation of the implementation of a mobile nursing information system. Online Journal of Nursing Informatics (OJNI), 23 (3), Available at http://www.himss.org/ojni (OJNI), 21(2).

The views and opinions expressed in this blog or by commenters are those of the author and do not necessarily reflect the official policy or position of HIMSS or its affiliates.

Online Journal of Nursing Informatics

Powered by the HIMSS Foundation and the HIMSS Nursing Informatics Community, the Online Journal of Nursing Informatics is a free, international, peer reviewed publication that is published three times a year and supports all functional areas of nursing informatics.

Read the Latest Edition