JMERC
Owner
Janbazan Medical and Engineering Research Center (JMERC) is the owner of IJWPH.
0.4
Cite Score
SJR: 0.114 / SNIP: 0.090
Volume 15, Issue 4 (2023)
Iran J War Public Health 2023, 15(4): 405-413 |
Back to browse issues page
Article Type:
Subject:
History
Received: 2023/12/6 | Accepted: 2024/01/9 | Published: 2024/01/17
Received: 2023/12/6 | Accepted: 2024/01/9 | Published: 2024/01/17
How to cite this article
Lorestani F, Zarghami M, Shatrian Mohamadi F. Designing Mental Health Indicators of Veterans Questionnaire Based on the Item Response Theory. Iran J War Public Health 2023; 15 (4) :405-413
URL: http://ijwph.ir/article-1-1420-en.html
URL: http://ijwph.ir/article-1-1420-en.html
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Rights and permissions
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
1- Department of Psychology, Saveh Branch, Islamic Azad University, Saveh, Iran
2- Behavioral Science Research Center, Lifestyle Research Institute, Tehran, Iran, Behavioral Science Research Center, Lifestyle Research Institute, Shikh Bahei Street, Tehran, Iran. Postal Code: 1435916471 (zar100@gmail.com)
2- Behavioral Science Research Center, Lifestyle Research Institute, Tehran, Iran, Behavioral Science Research Center, Lifestyle Research Institute, Shikh Bahei Street, Tehran, Iran. Postal Code: 1435916471 (zar100@gmail.com)
Full-Text (HTML) (197 Views)
Introduction
Development processes are considered one of any country's main and fundamental goals. There have always been criteria for achieving sustainable development that policymakers have focused on. Attaining a desirable level of mental health for individuals in society is the most important criterion for sustainable development in any country. As the World Health Organization has stated, ‘Achieving sustainable development without health will not be possible [1]. Mental health problems, especially mental disorders, usually increase mortality rates by up to 50%. Consequently, it significantly adds to the cost of physical health care. The high prevalence of mental health problems is a significant factor in increasing healthcare costs and changes in national economies, which poses a major obstacle to sustainable development. It imposes an economic burden on society while adversely affecting human resources [2].
On the other hand, society's health relies on providing appropriate and timely services. Many health problems and the slow progress of development programs stem from the lack of defining indicators for monitoring the achievement of goals [3]. Health indicators provide a real picture of community health. They reflect current conditions and measure the change process [4, 5, 6].
Mental health is one of the crucial indicators of individual health and sustainable development in any country. The Sustainable Development Goals of the United Nations are being pursued as a ‘shared plan’ aimed at creating a fairer and more sustainable world on a global scale. Mental health is a fundamental pillar of these initiatives; as the World Health Organization has stated, ‘Without mental health, there can be no health or sustainable development [2].
The mental health index serves as an evaluative variable of the mental health status under consideration; this index is a variable related to mental health and defines a subject in this area. Valid mental health indicators are important and essential for assessing and improving the quality of mental health programs because they can express developments in terms of the progress of mental health services [7]. Therefore, mental health indicators are important for monitoring the mental health status in the country, expanding mental health policies and programs, and evaluating the performance of these programs. Additionally, mental health indicators are necessary to compare and determine benchmarks [8].
Indicators are a common language among managers at various levels, managers and the community, policymakers, and implementers. Indicator-based management can be an effective management approach that will drive indicators, policies, resource allocation, and implementation plans [9]. The aim of developing indicators is to help understand the performance of health services, generate health information, and annually determine new indicators based on health service goals, reflect changes, and follow up on performance according to the plans [10].
Those involved in mental health and the planners require indicators for evaluating mental health [Flynn et al., 1996]. Therefore, one of the tools that mental health stakeholders in any organization need is mental health indicators to enable them to determine the status of the mental health of the population they serve and which path leads toward a better future. They can then monitor the mental health status within their target group and manage policies, implementation plans, resource allocation, and intervention evaluation accordingly. The healthcare system's comprehensive and complete data collection on the population and their health status is crucial for formulating appropriate policies, interventions, and planning aligned with the community’s needs [11]. Therefore, database information can be used to develop and generate indicators to determine health status. Without generating indicators, the healthcare system may wander, and resources may be inefficiently allocated without considering real needs.
The lack or absence of mental health indicators underscores the importance of studying decisions and planning based on the information system. One of the measurement theories used for developing and evaluating the psychometric properties of survey questionnaires with item response options such as Likert scales is the Item Response Theory (IRT). IRT is a powerful modeling approach [12].
The question-response framework utilizes various models to develop and assess the characteristics of Likert-type (multiple-choice or multiple-value) questionnaires. Multi-dimensional models of IRT will help us better understand what a question measures and determine the efficiency level of a respondent in each attribute. It even determines the accuracy with which different combinations measure abilities. Sigma (graded response) and partial score models are used to determine the parameters of questions and the respondent, which are suitable for multiple-value questionnaires [13]. Due to its capabilities, the IRT can be utilized in developing, validating, and evaluating the psychometric properties and indicators of mental health questionnaires.
As designing and implementing an efficient information system is a priority for any organization, evidence-based decision-making is crucial in reducing suffering, distress, and pain among veterans. Therefore, determining mental health indicators is unavoidable, and examining mental health indicators in veterans, in particular, is important. The present study aimed to develop mental health indicators for veterans using the IRT as one of the new measurement theories.
Materials and Methods
This is an instrumental psychometric study. Data collection is one of the important stages of research that requires appropriate tools [14]. Waltz et al. outline four steps in designing a suitable tool: 1- Providing a conceptual definition that the tool is intended to measure. 2- Determining the measurement objectives or conceptual domains. 3- Identifying more specific sub-domains of the concept to be measured and ultimately determining an appropriate number. Constructing the tool involves applying the above steps, preparing items, and scoring rules [15].
The findings of the first phase showed that components of quality of life (SF-12), components of psychiatric disorder checklists (SCL-90), and level of independence and dependence in daily activities (Bartel) were among the most determining factors of the model of veterans’ mental health. Physical performance, bodily pain, physical problems, vitality, social functioning, physical health, aggression, fear of illness, and sensitivity in interpersonal relationships were deemed more important and were used to determine the model of mental health. The demographic parameters included were the number of children, age, percentage of disability, percentage of nervous and mental disorders, psychiatric disorders, hospitalization history, type of hospitalization (general or psychiatric), and employment status [16]. The psychiatric disorder checklist (SCL-90) is a general tool for assessing the health status of individuals aged 14 and above. It includes subscales for physical complaints, obsessive-compulsive symptoms, sensitivity in interpersonal relationships, depression, anxiety, aggression, fear of illness, paranoia, and psychoticism. The reliability of this tool has been reported to be over 90% [17]. The quality-of-life questionnaire assesses physical functioning, physical role, bodily pain, general health, social functioning, emotional role, vitality, and mental health. A higher score indicates a better quality of life. Montazeri and colleagues have examined the validity of the questionnaire. In Iran, Cronbach’s alpha coefficient was reported to be 0.73 for the physical component summary and 0.72 for the mental component summary [18]. The Bartel questionnaire evaluates an individual’s level of independence in performing daily activities such as self-care, eating, and bathing based on the assistance they receive from others. The total score of each individual indicates their level of independence or dependence on daily activities. This tool consists of 10 questions, and the individual's level of dependence or independence is rated on a scale of 0, 2, 5, 8, and 10. A score of 10 indicates complete independence, while 0 indicates disability in the activity. This questionnaire was examined in various age groups of healthy individuals and patients, and its reliability was reported to be between 60 and 93% in different countries [19].
Two versions were prepared to prepare the questionnaire. The question designer prepared the first version, in which the desired dimensions were identified, and each dimension was defined. After defining the dimensions, the items of the initial questionnaire were prepared and arranged into a 40-item questionnaire using a Likert scale (ranging from ‘strongly disagree’ to ‘strongly agree’). In the next stage, the questionnaire was provided to 5 experts in the field of psychology, who were asked to identify the suitability or unsuitability of the items. All experts confirmed all 40 items and no items were removed at this stage. The study population consisted of veterans registered in the Comprehensive Information System for Veterans (Sajayya). Since the number of study variables was 17, and based on scientific references (sample size is usually twenty times the number of variables or 500 for factor analysis), a sample size of 500 was considered. Therefore, 500 veterans were randomly selected from the Sajayya system. Out of the 500 selected, 474 veterans participated in the study. In the next stage, the questionnaire was sent as a link to the 500 veterans, and they were asked to respond to the questionnaire.
The statistical analysis of the data consisted of two descriptive and inferential statistics. The descriptive statistics section investigated the indices such as mean, standard deviation, skewness, and kurtosis of the examined questions. The DETECT statistic and the DETECT index were initially used to identify the dimensionality in the data by the inferential statistics section. Using the ‘sirt’ package in the R software, the DETECT method was applied to determine the dimensions. Many measurement tools, especially attitude and personality assessment, utilize multiple-choice questions arranged in multiple categories. For this type of data, multi-category IRT models are needed to provide a nonlinear relationship between the trait level of the examinee and the probability of responding to one of the categories. Many multi-category item response theory models exist. This study utilized the Graded Response Model (GRM) by Samejima in 1996 and the Generalized Partial Credit Model (GPCM) by Muraki in 1993, which is suitable for multi-category data. The fitting of the model with the data and their comparison were assessed using the indices of relative likelihood fit ratio, Akaike’s Information Criterion (AIC), and Bayesian Information Criterion (BIC). Fit indices indicate the amount of information lost by the model. The lower the values of BIC and AIC, the better and more appropriate the model is compared to other models. The psychometric properties of the questions, namely the threshold and slope coefficients, were also estimated. The fitting indices of the questions were evaluated using the infit statistic, and individual fitting was assessed using the outfit statistic. The reliability of the questionnaire was evaluated using the empirical reliability index method. Test information and item characteristic curves were also examined. Data was analyzed using R 4.2.0 (packages mirt, Psych, sirt) and SPSS 26 software.
Findings
All 474 participating veterans were male, with a mean age of 53.4±35.2 years. Four hundred forty-eight veterans had children, and the mean number of children was 3.1±1.5. The average disability was 25.2±15.6%. Two hundred sixty-six veterans (56.2%) had psychological disorders, while 208 (43.88%) had psychological disorders in addition to other physical disabilities (Table 1).
Table 1. Demographic characteristics of the study participants
The mean scores of the items in the mental health index questionnaire ranged between 2.03±0.99 and 3.92±1.47, with the kurtosis between 0.86 and 0.97 and skewness between 1.38 and 0.04.
The DETECT test showed the dimensions ranging from two to eight scenarios, with an index between 4.86 and 6.47. The four-dimensional model showed more desirable values (Table 2).
Table 2. DETECT Test for very simple structure
The analysis and comparison of different models showed that the four-dimensional graded response model fits the data better. The four-dimensional graded response model, with BIC and AIC values of 85.48850 and 47.50157, respectively, had lower values compared to these indices in the two-, three-, and five-dimensional models (p<0.05). Additionally, the ratio of likelihood tests obtained (-24111.43) was higher than other models. Moreover, in comparison between the graded response model and the partial credit model, it was evident that the graded response model outperforms the partial credit model in each fit index, such as the likelihood ratio test, Akaike’s Information Criterion, and Bayesian Information Criterion, indicating its superiority. Therefore, considering the summarized results and the content validity of the items with the factors, the four-dimensional model was considered as the latent structure of the questionnaire (Table 3).
Table 3. Comparison of different models
The questions of the questionnaire were determined based on a four-factor model. The identified factors consist of neuroticism, personal competence, social functioning, and physical complaints, allocating 15, 6, 6, and 7 questions to each. For example, in the first factor identified as neuroticism, questions such as "Do you sometimes feel like breaking things?" and "When something breaks, do you blame others?" are included. Questions from the second factor, personal competence, include inquiries like "Can you eat alone?" and "Can you bathe alone?" Questions like "Do you feel as calm as you used to?" pertain to the social functioning dimension. Examining the questions of the fourth dimension, physical complaints, one can find inquiries such as "Do you feel weaker than before?" and "Have you felt dizzy in the past month?" It is also worth mentioning that 6 out of the 40 prepared questions were excluded from the questionnaire as they did not load significantly on any of the four dimensions. The analysis was conducted again after removing these six questions from the questionnaire. Then, the slope and threshold coefficients of the questions were examined. In terms of slope coefficients, each of the questions has values ranging from 0.3 to 2. Regarding the threshold coefficients, most questions fall from -4 to 4. The empirical reliability coefficient was used to assess the internal consistency reliability of the mental health questionnaire (Table 4).
Table 4. Empirical reliability of the dimensions of the mental health questionnaire in veterans
The characteristic curve of the neuroticism dimension, consisting of 15 questions, indicated that all 15 questions were affirmed. There was a suitable level of information at most ability levels (ability ranging from negative to positive 3), with the highest level of information falling within the ability range between negative and positive 2 (Figure 1).
Figure 1. Information curve of neuroticism dimension questions
All six questions of the general personal capacity dimension were confirmed, with the highest level of information falling within the range of abilities between negative 2 and positive 3 (Figure 2).
Figure 2. Information curve of the personal capacity dimension questions
All six questions of the social functioning dimension were confirmed, with the highest level of information between negative 3 and positive 2 (Figure 3).
Figure 3. Information curve of the social functioning questions
All seven questions of the physical complaints were confirmed, with the highest level of information between negative 3 and positive 2 (Figure 4).
Figure 4. Information curve of the social functioning questions
The overall information on the four dimensions (neuroticism, personal competence, social functioning, and physical complaints) indicated that the neuroticism dimension covers the entire range of abilities compared to the other three dimensions, and it almost covered the entire space between negative and positive 3. Regarding the dimensions of social functioning and physical complaints, the space between negative 3 and positive 2 encompasses the highest level of information. In the personal competence dimension, the range between negative 2 and positive 3 has the highest level of information. Comparing the level of information of the four dimensions within the range between negative 2 and positive 2, it can be said that neuroticism, personal competence, social functioning, and physical complaints provide information levels of 0.74, 0.94, 0.46, and 0.51, respectively (Figure 5).
Figure 5. Information curve of each of the four dimensions
Discussion
In this study, for the first time, we focused on constructing a tool for measuring the psychological health indicators of veterans using Item Response Theory (IRT). The major advantage of using this theory is the independence of questions from the sample group, meaning that the questions are not dependent on the sample group. This study examined the psychometric properties of the questionnaire under study. The results showed that the questionnaire had four dimensions of veterans’ psychological health indicators. These dimensions included neuroticism, personal competence, social functioning, and physical complaints.
The dimensions obtained from the psychological health questionnaire of veterans align with the Goldberger and Keas psychological health questionnaires. The Goldberger psychological health questionnaire comprises four subscales. The first subscale includes items related to individuals’ feelings about their health status and fatigue, which corresponds to the dimension of physical complaints in veterans’ psychological health indicators. The second subscale includes items related to anxiety and insomnia, which aligns with the dimension of neuroticism in the present questionnaire. The third subscale measures individuals’ ability to cope with professional demands and everyday life issues, assessing their feelings about handling common situations. This subscale is consistent with the dimensions of personal competence and social functioning in veterans’ psychological health indicators [18].
Additionally, the dimensions of this study align with the dimensions of emotional well-being, mental health, and social health of the Keas psychological health questionnaire [17]. The dimensions of personal competence and social functioning correlate with the items of the Bartel questionnaire and the subscales of quality of life [18, 19]. Furthermore, the dimension of physical complaints in the current questionnaire is consistent with the same dimension in the quality-of-life questionnaire and the psychiatric disorders checklist [17, 19].
In neuroticism, components related to aggressive behaviors, sensitivity in interpersonal relationships, and pathological fears are included. This finding is consistent with emotional regulation, which refers to the ability to recognize, express, and regulate one’s emotions as an important aspect of mental health [20]. Additionally, Rahimi demonstrated that the indicators of psychological well-being in Iranian elders consist of six dimensions, one of which is psychosocial. In this dimension, the absence of depression, psychiatric disorders, and personality disorders are part of the components, which is consistent with the dimension of neuroticism in the current study [20]. Furthermore, Mikaeli et al. stated that most veterans do not have satisfactory emotional, social, and health adaptation [21].
The dimension of personal competence is another aspect of the psychological well-being of veterans. This dimension includes components related to independence in daily activities. Wang et al. suggest that performing daily tasks and the level of dependence on others fall under the category of physical functioning, which aligns with the significant role of this variable in the indicators of psychological well-being among veterans [22]. Research findings have shown that social functioning is one of the dimensions of psychological well-being in veterans. Social functioning is one of the components of quality of life that plays a very important role in individuals’ psychological well-being. This finding is consistent with the statement by Liu et al., who assert that quality of life is a subset of emotional well-being [23]. Additionally, Lorestani states that quality of life variables are the most important determinants of the psychological well-being of veterans [16].
The dimension of physical complaints is one of the dimensions derived from the current research findings. Psychological well-being is an integral part of overall health and well-being and is important in maintaining physical health [24]. As Wang et al. suggest, individuals’ health, well-being, and physical, functional, social, psychological, and emotional well-being play a crucial role in their overall welfare and social prosperity. On the other hand, physical complaints are highlighted as one of the components discussed in the quality of life [25].
Attention to the mental health indicators of veterans was one of the strengths of this study, which had not been addressed in any previous investigation or program. Additionally, the examination was conducted across all veteran groups without considering any specific percentage or type of veteran. The advantage of this study over previous research lies in developing a tool that can assess the mental health indicators of veterans. On the other hand, using the new theory of measurement, IRT (Item Response Theory), was another advantage of this study. However, this study had limitations and weaknesses. In this study, indicators were only determined within the veteran group, and the inclusion of their family members and experts was not considered.
Since one of the responsibilities of every organization and institution is to pay attention to mental health and promote it within their covered group, it is recommended to consider the indicators obtained from the present study for evidence-based decision-making. Furthermore, it is suggested that the obtained indicators be used to evaluate past performance, compare it with the current situation, assess the goals of development programs, and compare the performance among different units. Researchers are also advised to consider mental health indicators separately for veterans and in groups of experts and family members of veterans in their future investigations.
Conclusion
The mental health indicators questionnaire for veterans comprises four dimensions and possesses high and adequate reliability. When disaggregated across each dimension of neuroticism, personal capability, social functioning, and physical complaints, the questionnaire items demonstrate discriminative power and information across different levels of ability, ranging between 3±. This means that it provides the necessary information regarding the mental health indicators of veterans at various levels of ability.
Acknowledgments: We are grateful to all those who have assisted in any way.
Ethical Permissions: This dissertation was conducted with the spiritual support of the Janbazan Research Institute under the ethical code IR.ISAAR.REC.1401.007.
Conflicts of Interests: There is no conflict of interest.
Authors’ Contribution: Lorestani F (First Author), Introduction Writer/Main Researcher/Discussion Writer (50%); Zarghami MH (Second Author), Statistical Analyst (30%); Shatrian Mohamadi F (Third Author), Methodologist/Assistant Researcher (20%);
Funding/Support: The article was extracted from the first author's doctoral dissertation on psychometrics.
Development processes are considered one of any country's main and fundamental goals. There have always been criteria for achieving sustainable development that policymakers have focused on. Attaining a desirable level of mental health for individuals in society is the most important criterion for sustainable development in any country. As the World Health Organization has stated, ‘Achieving sustainable development without health will not be possible [1]. Mental health problems, especially mental disorders, usually increase mortality rates by up to 50%. Consequently, it significantly adds to the cost of physical health care. The high prevalence of mental health problems is a significant factor in increasing healthcare costs and changes in national economies, which poses a major obstacle to sustainable development. It imposes an economic burden on society while adversely affecting human resources [2].
On the other hand, society's health relies on providing appropriate and timely services. Many health problems and the slow progress of development programs stem from the lack of defining indicators for monitoring the achievement of goals [3]. Health indicators provide a real picture of community health. They reflect current conditions and measure the change process [4, 5, 6].
Mental health is one of the crucial indicators of individual health and sustainable development in any country. The Sustainable Development Goals of the United Nations are being pursued as a ‘shared plan’ aimed at creating a fairer and more sustainable world on a global scale. Mental health is a fundamental pillar of these initiatives; as the World Health Organization has stated, ‘Without mental health, there can be no health or sustainable development [2].
The mental health index serves as an evaluative variable of the mental health status under consideration; this index is a variable related to mental health and defines a subject in this area. Valid mental health indicators are important and essential for assessing and improving the quality of mental health programs because they can express developments in terms of the progress of mental health services [7]. Therefore, mental health indicators are important for monitoring the mental health status in the country, expanding mental health policies and programs, and evaluating the performance of these programs. Additionally, mental health indicators are necessary to compare and determine benchmarks [8].
Indicators are a common language among managers at various levels, managers and the community, policymakers, and implementers. Indicator-based management can be an effective management approach that will drive indicators, policies, resource allocation, and implementation plans [9]. The aim of developing indicators is to help understand the performance of health services, generate health information, and annually determine new indicators based on health service goals, reflect changes, and follow up on performance according to the plans [10].
Those involved in mental health and the planners require indicators for evaluating mental health [Flynn et al., 1996]. Therefore, one of the tools that mental health stakeholders in any organization need is mental health indicators to enable them to determine the status of the mental health of the population they serve and which path leads toward a better future. They can then monitor the mental health status within their target group and manage policies, implementation plans, resource allocation, and intervention evaluation accordingly. The healthcare system's comprehensive and complete data collection on the population and their health status is crucial for formulating appropriate policies, interventions, and planning aligned with the community’s needs [11]. Therefore, database information can be used to develop and generate indicators to determine health status. Without generating indicators, the healthcare system may wander, and resources may be inefficiently allocated without considering real needs.
The lack or absence of mental health indicators underscores the importance of studying decisions and planning based on the information system. One of the measurement theories used for developing and evaluating the psychometric properties of survey questionnaires with item response options such as Likert scales is the Item Response Theory (IRT). IRT is a powerful modeling approach [12].
The question-response framework utilizes various models to develop and assess the characteristics of Likert-type (multiple-choice or multiple-value) questionnaires. Multi-dimensional models of IRT will help us better understand what a question measures and determine the efficiency level of a respondent in each attribute. It even determines the accuracy with which different combinations measure abilities. Sigma (graded response) and partial score models are used to determine the parameters of questions and the respondent, which are suitable for multiple-value questionnaires [13]. Due to its capabilities, the IRT can be utilized in developing, validating, and evaluating the psychometric properties and indicators of mental health questionnaires.
As designing and implementing an efficient information system is a priority for any organization, evidence-based decision-making is crucial in reducing suffering, distress, and pain among veterans. Therefore, determining mental health indicators is unavoidable, and examining mental health indicators in veterans, in particular, is important. The present study aimed to develop mental health indicators for veterans using the IRT as one of the new measurement theories.
Materials and Methods
This is an instrumental psychometric study. Data collection is one of the important stages of research that requires appropriate tools [14]. Waltz et al. outline four steps in designing a suitable tool: 1- Providing a conceptual definition that the tool is intended to measure. 2- Determining the measurement objectives or conceptual domains. 3- Identifying more specific sub-domains of the concept to be measured and ultimately determining an appropriate number. Constructing the tool involves applying the above steps, preparing items, and scoring rules [15].
The findings of the first phase showed that components of quality of life (SF-12), components of psychiatric disorder checklists (SCL-90), and level of independence and dependence in daily activities (Bartel) were among the most determining factors of the model of veterans’ mental health. Physical performance, bodily pain, physical problems, vitality, social functioning, physical health, aggression, fear of illness, and sensitivity in interpersonal relationships were deemed more important and were used to determine the model of mental health. The demographic parameters included were the number of children, age, percentage of disability, percentage of nervous and mental disorders, psychiatric disorders, hospitalization history, type of hospitalization (general or psychiatric), and employment status [16]. The psychiatric disorder checklist (SCL-90) is a general tool for assessing the health status of individuals aged 14 and above. It includes subscales for physical complaints, obsessive-compulsive symptoms, sensitivity in interpersonal relationships, depression, anxiety, aggression, fear of illness, paranoia, and psychoticism. The reliability of this tool has been reported to be over 90% [17]. The quality-of-life questionnaire assesses physical functioning, physical role, bodily pain, general health, social functioning, emotional role, vitality, and mental health. A higher score indicates a better quality of life. Montazeri and colleagues have examined the validity of the questionnaire. In Iran, Cronbach’s alpha coefficient was reported to be 0.73 for the physical component summary and 0.72 for the mental component summary [18]. The Bartel questionnaire evaluates an individual’s level of independence in performing daily activities such as self-care, eating, and bathing based on the assistance they receive from others. The total score of each individual indicates their level of independence or dependence on daily activities. This tool consists of 10 questions, and the individual's level of dependence or independence is rated on a scale of 0, 2, 5, 8, and 10. A score of 10 indicates complete independence, while 0 indicates disability in the activity. This questionnaire was examined in various age groups of healthy individuals and patients, and its reliability was reported to be between 60 and 93% in different countries [19].
Two versions were prepared to prepare the questionnaire. The question designer prepared the first version, in which the desired dimensions were identified, and each dimension was defined. After defining the dimensions, the items of the initial questionnaire were prepared and arranged into a 40-item questionnaire using a Likert scale (ranging from ‘strongly disagree’ to ‘strongly agree’). In the next stage, the questionnaire was provided to 5 experts in the field of psychology, who were asked to identify the suitability or unsuitability of the items. All experts confirmed all 40 items and no items were removed at this stage. The study population consisted of veterans registered in the Comprehensive Information System for Veterans (Sajayya). Since the number of study variables was 17, and based on scientific references (sample size is usually twenty times the number of variables or 500 for factor analysis), a sample size of 500 was considered. Therefore, 500 veterans were randomly selected from the Sajayya system. Out of the 500 selected, 474 veterans participated in the study. In the next stage, the questionnaire was sent as a link to the 500 veterans, and they were asked to respond to the questionnaire.
The statistical analysis of the data consisted of two descriptive and inferential statistics. The descriptive statistics section investigated the indices such as mean, standard deviation, skewness, and kurtosis of the examined questions. The DETECT statistic and the DETECT index were initially used to identify the dimensionality in the data by the inferential statistics section. Using the ‘sirt’ package in the R software, the DETECT method was applied to determine the dimensions. Many measurement tools, especially attitude and personality assessment, utilize multiple-choice questions arranged in multiple categories. For this type of data, multi-category IRT models are needed to provide a nonlinear relationship between the trait level of the examinee and the probability of responding to one of the categories. Many multi-category item response theory models exist. This study utilized the Graded Response Model (GRM) by Samejima in 1996 and the Generalized Partial Credit Model (GPCM) by Muraki in 1993, which is suitable for multi-category data. The fitting of the model with the data and their comparison were assessed using the indices of relative likelihood fit ratio, Akaike’s Information Criterion (AIC), and Bayesian Information Criterion (BIC). Fit indices indicate the amount of information lost by the model. The lower the values of BIC and AIC, the better and more appropriate the model is compared to other models. The psychometric properties of the questions, namely the threshold and slope coefficients, were also estimated. The fitting indices of the questions were evaluated using the infit statistic, and individual fitting was assessed using the outfit statistic. The reliability of the questionnaire was evaluated using the empirical reliability index method. Test information and item characteristic curves were also examined. Data was analyzed using R 4.2.0 (packages mirt, Psych, sirt) and SPSS 26 software.
Findings
All 474 participating veterans were male, with a mean age of 53.4±35.2 years. Four hundred forty-eight veterans had children, and the mean number of children was 3.1±1.5. The average disability was 25.2±15.6%. Two hundred sixty-six veterans (56.2%) had psychological disorders, while 208 (43.88%) had psychological disorders in addition to other physical disabilities (Table 1).
Table 1. Demographic characteristics of the study participants
The mean scores of the items in the mental health index questionnaire ranged between 2.03±0.99 and 3.92±1.47, with the kurtosis between 0.86 and 0.97 and skewness between 1.38 and 0.04.
The DETECT test showed the dimensions ranging from two to eight scenarios, with an index between 4.86 and 6.47. The four-dimensional model showed more desirable values (Table 2).
Table 2. DETECT Test for very simple structure
The analysis and comparison of different models showed that the four-dimensional graded response model fits the data better. The four-dimensional graded response model, with BIC and AIC values of 85.48850 and 47.50157, respectively, had lower values compared to these indices in the two-, three-, and five-dimensional models (p<0.05). Additionally, the ratio of likelihood tests obtained (-24111.43) was higher than other models. Moreover, in comparison between the graded response model and the partial credit model, it was evident that the graded response model outperforms the partial credit model in each fit index, such as the likelihood ratio test, Akaike’s Information Criterion, and Bayesian Information Criterion, indicating its superiority. Therefore, considering the summarized results and the content validity of the items with the factors, the four-dimensional model was considered as the latent structure of the questionnaire (Table 3).
Table 3. Comparison of different models
The questions of the questionnaire were determined based on a four-factor model. The identified factors consist of neuroticism, personal competence, social functioning, and physical complaints, allocating 15, 6, 6, and 7 questions to each. For example, in the first factor identified as neuroticism, questions such as "Do you sometimes feel like breaking things?" and "When something breaks, do you blame others?" are included. Questions from the second factor, personal competence, include inquiries like "Can you eat alone?" and "Can you bathe alone?" Questions like "Do you feel as calm as you used to?" pertain to the social functioning dimension. Examining the questions of the fourth dimension, physical complaints, one can find inquiries such as "Do you feel weaker than before?" and "Have you felt dizzy in the past month?" It is also worth mentioning that 6 out of the 40 prepared questions were excluded from the questionnaire as they did not load significantly on any of the four dimensions. The analysis was conducted again after removing these six questions from the questionnaire. Then, the slope and threshold coefficients of the questions were examined. In terms of slope coefficients, each of the questions has values ranging from 0.3 to 2. Regarding the threshold coefficients, most questions fall from -4 to 4. The empirical reliability coefficient was used to assess the internal consistency reliability of the mental health questionnaire (Table 4).
Table 4. Empirical reliability of the dimensions of the mental health questionnaire in veterans
The characteristic curve of the neuroticism dimension, consisting of 15 questions, indicated that all 15 questions were affirmed. There was a suitable level of information at most ability levels (ability ranging from negative to positive 3), with the highest level of information falling within the ability range between negative and positive 2 (Figure 1).
Figure 1. Information curve of neuroticism dimension questions
All six questions of the general personal capacity dimension were confirmed, with the highest level of information falling within the range of abilities between negative 2 and positive 3 (Figure 2).
Figure 2. Information curve of the personal capacity dimension questions
All six questions of the social functioning dimension were confirmed, with the highest level of information between negative 3 and positive 2 (Figure 3).
Figure 3. Information curve of the social functioning questions
All seven questions of the physical complaints were confirmed, with the highest level of information between negative 3 and positive 2 (Figure 4).
Figure 4. Information curve of the social functioning questions
The overall information on the four dimensions (neuroticism, personal competence, social functioning, and physical complaints) indicated that the neuroticism dimension covers the entire range of abilities compared to the other three dimensions, and it almost covered the entire space between negative and positive 3. Regarding the dimensions of social functioning and physical complaints, the space between negative 3 and positive 2 encompasses the highest level of information. In the personal competence dimension, the range between negative 2 and positive 3 has the highest level of information. Comparing the level of information of the four dimensions within the range between negative 2 and positive 2, it can be said that neuroticism, personal competence, social functioning, and physical complaints provide information levels of 0.74, 0.94, 0.46, and 0.51, respectively (Figure 5).
Figure 5. Information curve of each of the four dimensions
Discussion
In this study, for the first time, we focused on constructing a tool for measuring the psychological health indicators of veterans using Item Response Theory (IRT). The major advantage of using this theory is the independence of questions from the sample group, meaning that the questions are not dependent on the sample group. This study examined the psychometric properties of the questionnaire under study. The results showed that the questionnaire had four dimensions of veterans’ psychological health indicators. These dimensions included neuroticism, personal competence, social functioning, and physical complaints.
The dimensions obtained from the psychological health questionnaire of veterans align with the Goldberger and Keas psychological health questionnaires. The Goldberger psychological health questionnaire comprises four subscales. The first subscale includes items related to individuals’ feelings about their health status and fatigue, which corresponds to the dimension of physical complaints in veterans’ psychological health indicators. The second subscale includes items related to anxiety and insomnia, which aligns with the dimension of neuroticism in the present questionnaire. The third subscale measures individuals’ ability to cope with professional demands and everyday life issues, assessing their feelings about handling common situations. This subscale is consistent with the dimensions of personal competence and social functioning in veterans’ psychological health indicators [18].
Additionally, the dimensions of this study align with the dimensions of emotional well-being, mental health, and social health of the Keas psychological health questionnaire [17]. The dimensions of personal competence and social functioning correlate with the items of the Bartel questionnaire and the subscales of quality of life [18, 19]. Furthermore, the dimension of physical complaints in the current questionnaire is consistent with the same dimension in the quality-of-life questionnaire and the psychiatric disorders checklist [17, 19].
In neuroticism, components related to aggressive behaviors, sensitivity in interpersonal relationships, and pathological fears are included. This finding is consistent with emotional regulation, which refers to the ability to recognize, express, and regulate one’s emotions as an important aspect of mental health [20]. Additionally, Rahimi demonstrated that the indicators of psychological well-being in Iranian elders consist of six dimensions, one of which is psychosocial. In this dimension, the absence of depression, psychiatric disorders, and personality disorders are part of the components, which is consistent with the dimension of neuroticism in the current study [20]. Furthermore, Mikaeli et al. stated that most veterans do not have satisfactory emotional, social, and health adaptation [21].
The dimension of personal competence is another aspect of the psychological well-being of veterans. This dimension includes components related to independence in daily activities. Wang et al. suggest that performing daily tasks and the level of dependence on others fall under the category of physical functioning, which aligns with the significant role of this variable in the indicators of psychological well-being among veterans [22]. Research findings have shown that social functioning is one of the dimensions of psychological well-being in veterans. Social functioning is one of the components of quality of life that plays a very important role in individuals’ psychological well-being. This finding is consistent with the statement by Liu et al., who assert that quality of life is a subset of emotional well-being [23]. Additionally, Lorestani states that quality of life variables are the most important determinants of the psychological well-being of veterans [16].
The dimension of physical complaints is one of the dimensions derived from the current research findings. Psychological well-being is an integral part of overall health and well-being and is important in maintaining physical health [24]. As Wang et al. suggest, individuals’ health, well-being, and physical, functional, social, psychological, and emotional well-being play a crucial role in their overall welfare and social prosperity. On the other hand, physical complaints are highlighted as one of the components discussed in the quality of life [25].
Attention to the mental health indicators of veterans was one of the strengths of this study, which had not been addressed in any previous investigation or program. Additionally, the examination was conducted across all veteran groups without considering any specific percentage or type of veteran. The advantage of this study over previous research lies in developing a tool that can assess the mental health indicators of veterans. On the other hand, using the new theory of measurement, IRT (Item Response Theory), was another advantage of this study. However, this study had limitations and weaknesses. In this study, indicators were only determined within the veteran group, and the inclusion of their family members and experts was not considered.
Since one of the responsibilities of every organization and institution is to pay attention to mental health and promote it within their covered group, it is recommended to consider the indicators obtained from the present study for evidence-based decision-making. Furthermore, it is suggested that the obtained indicators be used to evaluate past performance, compare it with the current situation, assess the goals of development programs, and compare the performance among different units. Researchers are also advised to consider mental health indicators separately for veterans and in groups of experts and family members of veterans in their future investigations.
Conclusion
The mental health indicators questionnaire for veterans comprises four dimensions and possesses high and adequate reliability. When disaggregated across each dimension of neuroticism, personal capability, social functioning, and physical complaints, the questionnaire items demonstrate discriminative power and information across different levels of ability, ranging between 3±. This means that it provides the necessary information regarding the mental health indicators of veterans at various levels of ability.
Acknowledgments: We are grateful to all those who have assisted in any way.
Ethical Permissions: This dissertation was conducted with the spiritual support of the Janbazan Research Institute under the ethical code IR.ISAAR.REC.1401.007.
Conflicts of Interests: There is no conflict of interest.
Authors’ Contribution: Lorestani F (First Author), Introduction Writer/Main Researcher/Discussion Writer (50%); Zarghami MH (Second Author), Statistical Analyst (30%); Shatrian Mohamadi F (Third Author), Methodologist/Assistant Researcher (20%);
Funding/Support: The article was extracted from the first author's doctoral dissertation on psychometrics.
Keywords:
References
1. Tedros AG. The WHO Special Initiative for Mental Health,2019-2023. Gevenva; WHO; 2019. [Link]
2. LayaRd R. The economics of mental health. IZA World of Labor. 2017. [Link] [DOI:10.15185/izawol.321]
3. Mohammad K, Ahmadi K, Fathi Ashtiani A, Azad Falla P, Ebadi A. Determining the mental health indicators in military personne. J Police Med. 2015;3(4):239-48. [Persian] [Link]
4. Sandoval C, Couris C, Leeb K. New mental health indicators provide a snapshot on performance of the mental health system in Canada. Healthc Q. 2012;15(2):14-6. [Link] [DOI:10.12927/hcq.2012.22919]
5. Grant-Pearce C, Miles I, Hills P. Mismatches in priorities for health research between professionals and consumers. Manchester: Policy Research in Engineering science and Technology, university of Manchester; 1998.
6. Wilkinson J, Bywaters J, Simms S, Chappel D, Glover G. Developing mental health indicators in England. Public Health. 2008;122(9): 897-905. [Link] [DOI:10.1016/j.puhe.2007.10.012]
7. Kotera Y, Conway E, Van Gordon W. Mental health of UK university business students: Relationship with shame, motivation and self-compassion. J Educ Bus. 2019;94(1):11-20. [Link] [DOI:10.1080/08832323.2018.1496898]
8. Parker LA, Sullivan S, Cacho N, Engelmann C, Krueger C, Mueller M. Indicators of secretory activation in mothers of preterm very low birth weight infants. J Hum Lact. 2021;37(3):581-92. [Link] [DOI:10.1177/0890334420980424]
9. Khosravi A, Najafi F, Rahbar,Motlagh M, Kabir J. Health image indicators in the Islamic Republic. 2009. [Persian]
10. Mental Health Commission of Canada. Information about the indicators and the Informing the Future [Internet]. Ottawa: Mental Health Indicators for Canada; 2015- [2015 July 1]. Available from: https://mentalhealthcommission.ca/resource/informing-the-future-mental-health-indicators-for-canada/ [Link]
11. Ministry of treatment and medical education. National Health Indicators. Tehran: Ministry of treatment and medical education; 2017. [Persian] [Link]
12. Baniasadi A, Salehi K, Khodaie E, Bagheri K. Fair classroom assessment rubric: Fitting a multidimensional graded response model. Q Educ Measurement. 2022;3(49):31-51. [Link]
13. Kabiri M, Moghadam A, Farzad V, Ahmadi M, Khodaei A, Ghotbedini M. Applying Bi-factor multidimensional item-response theory model for dimensionality and differential items functioning analysis on testlet-based tests. Q Educ Measurement. 2017;7(28):121-53. [Persian] [Link]
14. Afrasiabifar A, Yaghmaei F, Abdoli S. Research tool translation and cross-cultural adaptation. Facul Nurs Midwif Q. 2006;16:58-67.
15. Waltz CF, Strickland OL, Lenz ER. Measurement in nursing and health research. 4th ed. New York: Springer Publishing Company; 2010. [Link]
16. Lorestani F, Zarghami MH, Shatrian F, Mosavi B. Mental health model of Iranian veterans with network analysis appriach. Iran J War Public Health. 2023;15(2):123-31. [Link]
17. Montazeri A, Vahdaninia M, Mousavi SJ, Omidvari S. The Iranian version of 12-item short form health survey (SF-12): Factor structure, internal consistency and construct validity. BMC Public Health. 2009;9:341 [Link] [DOI:10.1186/1471-2458-9-341]
18. FathiAshtiani A. Psychological Tese Personality and Mentan Health. Tehran: Besaat; 2014. p.39-60. [Link]
19. Mansfield KE, Mathur R, Tazare J, Henderson AD, Mulick AR, Carreira H, et al. Indirect acute effects of the COVID-19 pandemic on physical and mental health in the UK. Lancet Digit Health. 2021;3(4):e217-30. [Link] [DOI:10.1016/S2589-7500(21)00017-0]
20. Rahimi A, Ahmadi K, Asgharnejad Fari A. Assessing the status of mental health indices in Iranian elderly people. J Psychol Sci. 2019;18(77):529-38. [Persian] [Link]
21. Mikaeili N, Einy S, Taghavy R. Role of coping styles, locus of control, and emotional intelligence in predicting social adjustment of veterans with post-traumatic stress disorder. Iran J War Public Health. 2018;10(4):195-201. [Link] [DOI:10.29252/ijwph.10.4.195]
22. Bjørlykhaug KI, Karlsson B, Hesook SK, Kleppe LC. Social support and recovery from mental health problems: A scoping review. Nordic Soc Work Res. 2021;12(5):666-97. [Link] [DOI:10.1080/2156857X.2020.1868553]
23. Liu X, Ping S, Gao W. Changes in undergraduate students' psychological well-being as they experience university life. Int J Environ Rese Public Health. 2019;16(16):2864. [Link] [DOI:10.3390/ijerph16162864]
24. Baiden P, Den Dunnen W, Fallon B. Examining the independent effect of social support on unmet mental healthcare needs among Canadians. Soc Indicators Res. 2017;130(3):1229-46. [Link] [DOI:10.1007/s11205-015-1224-y]
25. Wang J, Mann F, Lloyd-Evans B, Ma R, Johnson S. Associations between loneliness and perceived social support and outcomes of mental health problems: A systematic review. BMC Psychiatry. 2018;18(156):1-16. [Link] [DOI:10.1186/s12888-018-1736-5]