1. Introduction
Severe mental disorders are like a double-edged sword, that one edge is the symptom distress and disability that prevents a person from achieving personal goals, and the other edge is the stigma and social injustice that many people with the label of mental disorder experience, which can be profoundly exhausting and challenging for the individual and those around him/her. Stigma is a universal phenomenon and the most important obstacle to seeking medical help. Although the quality and effectiveness of treatment and mental health services have greatly improved over the past 50 years, even the therapeutic revolutions in psychiatry have not yet been able to eliminate the stigma of mental disorders. Mental disorder is not shameful and just like heart disease, is a medical problem (Stuart, 2018).
Stigma includes negative attitudes toward someone that occur based on a distinctive characteristic, such as race, sexual orientation, disability, or mental disorder. Stigma leads to prejudice, and prejudice is an emotional reaction that leads to discrimination. Discrimination means behaving negatively because of the same specific characteristics (Bos et al., 2017).
Stigma is usually caused by a lack of knowledge and stereotypes. The consequences of stigma can be humiliating and life-threatening and diminish a person’s human dignity, and leads to marginalization and neglect of the person. It can cause significant economic problems for the person and deprive him/her of his basic needs and even lead to suicide (Corrigan et al., 2018).
People with mental illness have long been affected by social stigma and perceived stigma. Mental illness has been a subject of negative judgment and stigmatization (O’Reilly & Lester, 2017).
Social stigma, also called the public stigma, refers to a set of prejudicial attitudes, discriminatory behaviors, and biased structures toward an individual or group of individuals based on the stigma-forming beliefs associated with the target group (Holley et al., 2017).
Self-stigma, also called internalized stigma or perceived stigma, occurs when a member of a stigmatized group internalizes negative social stereotypes about the group (Lucksted & Drapalski, 2018).
Regarding the negative consequences of the stigma in the course of the disorder and the life of people with mental illness, this study was done to investigate the psychometric properties of the Iranian version of the Stig-9 questionnaire in people with mental disorders.

2. Materials and Methods
The present study was descriptive and standardized. The statistical population of this study included all military personnel with mental illnesses, who were hospitalized in the 505 Psychiatric Hospital from October 2020 to the end of January 2021. The sample size was estimated to be at least 95 people, based on the formula“ 50+5n”, where “n” was the number of questions in the questionnaire (Kim & Park, 2019). 
Inclusion criteria:
1. Age ≤18 years
2. Being literate
3. The diagnosis of mental illness based on structured clinical interviews and the DSM-5 and psychiatrist diagnosis of the 505 Psychiatric Hospital.
Exclusion criteria:
1. Having psychotic disorders
2. Incomplete questionnaire
Sampling: In this study, convenience sampling was performed in two stages as follows:
To evaluate the face and content validity, 15 experts were selected, including ten psychologists and five psychiatrists.
To evaluate the convergent validity and internal consistency, 95 samples were selected from patients admitted to the 505 Psychiatric Hospital (n=95).
Research instruments
1. The researcher-made demographic information questionnaire:
2. The Stig-9 questionnaire:
The Stig-9 questionnaire is a self-assessment questionnaire, developed by Gierk et al. (2018) in Hamburg to assess the perceived stigma of mentally ill patients. It is a short questionnaire that measures the cognitive, behavioral, and emotional dimensions of perceived stigma in mentally ill cases (α=0.88). This questionnaire is based on the modified labeling theory (link, 1989). It has nine items and three subscales, including discrimination, negative emotions, and labeling. The answers are on a 4-point Likert scale (disagree, somewhat disagree, somewhat agree, and agree). The score ranges from 0 to 27, and the higher scores indicate stronger expectations for negative social beliefs, feelings, and behaviors toward mental illness. In a study, men showed a higher level of perceived stigma (mean score: 41±14, n=919, 31% male) (Gierk et al., 2018). There are currently a German version (original version) and an English version of this questionnaire. Adeyemi et al. (2018) standardized the English version of the questionnaire in Kaduna, Northern Nigeria. They found that women all under the age of 31, showed a higher level of perceived stigma (α=0.71, n=120, mean score: 11.33±35.47) (Adeymi et al., 2018).
Procedure:
1. After correspondence with the developers and obtaining permission from them, the questionnaire was translated from English to Persian by two translators considering scientific aspects, then, a single copy of the above two translated questionnaires was obtained by the researcher, considering the best translation of each item.
2. The questionnaire was reviewed by a team of 15 experts, including ten psychologists and five psychiatrists in terms of face and content validity. Impact score, content validity ratio (CVR), and content validity index (CVI) were calculated based on their opinions and changes were applied to the Persian version of the Stig-9 questionnaire according to the expert’s ideas.
3. In the next step, the prepared questionnaire was performed on a sample of 20 people, and the minor problems were corrected.
4. The back translation of the questionnaire was done by two other translators from Persian to English, and then, the two prepared files were merged. The back-translated questionnaire, which corresponded to 75% of the English version of the questionnaire, was sent to the developers and approved by them.
5. The finalized Persian version of the questionnaire was performed on people with mental illness admitted to the 505 Psychiatric Hospital based on inclusion and exclusion criteria.
In this study, descriptive and inferential statistics were used to analyze the data using SPSS Software, version 26. For factor analysis, R Coding software version 3.6.3 and R package lavaan version 0.7 were used.
In this study, Cronbach’s α coefficient was used to determine internal consistency. The face and content validity were assessed simultaneously. The face validity was assessed quantitatively, using the impact score. To determine face validity, the questionnaire was given to the experts to have an impact score 
Content validity was assessed quantitatively, using CVR and CVI. The CVR was calculated according to Lawshe’s formula (1975) and based on the experts’ opinions, the minimum acceptable amount of CVR was determined to be 0.49 and items withwere modified or removed (Baghestani et al., 2019).
Waltz and Baussel’s formula (1981) was used to assess the CVI. Hyrkas e al. (2003) recommended a to accept each item (Almanasreh et al., 2019). Construct validity was assessed by exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). To investigate the internal correlation of the questionnaire, convergent validity was calculated, using average variance extracted (AVE) and composite reliability (CR).

3. Results
The sample included military personnel with mental disorders that were hospitalized in the 505 Psychiatric Hospital from October 2020 to the end of January 2021, of whom 73 cases (%76.8) were male and 43 cases (%45.3) were single. Also, 48 cases (%50.5) were self-employed. Six cases (%6.3) were students, 29 cases (%30.5) were employees, and 12 cases (%12.6) were unemployed. Also, 19 samples (%20) dropped out of high school, 40(%42.1) had a diploma, eight cases (%8.4) had an associate degree, 22 cases (%23.2), had a bachelor’s degree, 4 cases (%4.2) had a masters’ degree, two cases (%2.1) had a PhD, and 36 participants (%38.5) had university educations. Sixteen samples (%16.8) (had a history of physical illness. 
Also, the youngest and oldest participants were 18 and 65 years old, respectively. Also, 36 people (%37.9) had major depressive disorder (MDD), 32 people (%33.7) had a borderline personality disorder (BPD), nine people (%9.5) had adjustment disorder, seven people (%7.4) had generalized anxiety disorder (GAD), six people (%6.3) had obsessive-compulsive disorder (OCD), four people (%4.2) had with panic disorder, and one person (%1.1) had anxiety disorder.
Reliability: Cronbach’s α coefficient was used to evaluate the reliability or internal consistency. The Cronbach’s α of the whole questionnaire was 0.86, and that of the discrimination subscale (items 1-3), negative emotions (items 4-6), and labeling (items 7-9) was 0.69 and 0.74, respectively. 
Validity
Face and content validity
Table 1 shows the face and content validity assessment of the questionnaire.
As Table 1 shows, in all items, the impact score was, and was thus, the face and content validity of all items was confirmed by the experts. 
Construct validity: The construct validity in this study was performed using EFA and CFA.



Exploratory factor analysis: To perform EFA, the Kaiser-Meyer-Olkin (KMO) test was performed to ensure the adequacy of the sample size. Then, Bartlett’s test of sphericity was used to determine that the correlation between the variable is not zero. The sampling adequacy index was equal to 0.854 and KMO was >0.6, and the sample size was suitable for factor analysis. Also, considering that Bartlett’s test was significant with a degree of freedom of 36 at the level of P<0.001, factor analysis was appropriate to identify factor structure. The result of exploratory factor analysis showed that the questionnaire has three factors that overall explained %70.1 of the total variance. Table 2 shows the rotated matrix of the domains, by the varimax method and includes the factor loading of the questions in each of the three components (discrimination, negative emotions, and labeling) after rotation. According to Table 2, item three is not included in its respective component; thus, this factor was removed from CFA. 



Confirmatory factor analysis: 
Table 3 shows the goodness of fit index (GFI) of CFA. According to Table 3, χ2 was 22.45, χ2/df was <5, SRMR was 0.90, RMSEA was 0.06, and Tucker-lewis index (TLI) and CFI were 0.99, which was higher than 0.90. Therefore, the model fit indices showed that the model had a good fit.



Table 4 shows that the standardized factor loading on all items was more than 0.4 and significant (P<0.05; therefore, all items, except item 3, had an effective and acceptable role in the formation of the respective component. Also, when evaluating the content validity, some experts were hesitant about remaining item 3 after analysis.



Convergent validity
According to Table 5, for all three components, AVE was >0.5, CR was >0.7, and CR>AVE. Thus, the convergent validity of the questionnaire was confirmed.



4. Discussion 
Stigma is a risk factor that directly and indirectly has many negative effects on the course of the disorder and various social, educational, and occupational aspects of the affected person’s life and the society. Some of these negative effects on a person with a mental disorder are exacerbation of symptoms of the disorder, unwillingness to seek medical help and being less likely to continue treatment, inadequate understanding of family, friends, or others, fewer opportunities for employment or activities, and reducing social support (Park, 2018).
 In addition to stigma, labeling is a social problem. People with mental disorders face many challenges in living in a society that does not understand them. These people are not socially included in the mainstream of society. Stigma is an inherent part of society’s social structures and exists in the way laws and social services are structured. Deprivation of many citizenship rights, social isolation, inadequate housing, unemployment, and poverty are all linked to mental health. Thus, stigma and discrimination can trap people in a vicious cycle (Corrigan & Kleinlein, 2018).
Due to these negative effects of stigma on the course of the disorder and the life of people with mental illness, the present study was conducted to investigate the psychometric properties of the Iranian version of the Stig-9 questionnaire in people with mental disorders.
Face and content validity were assessed quantitatively and according to the opinion of the experts, face validity was calculated according to the impact score formula, with an impact score of >1.5; thus, all items were confirmed in terms of face validity. Gierk et al. (2018) examined face validity qualitatively, due to the psychological effect of the wording. Adeymi et al. (2018) did not examine the face validity of the questionnaire. Content validity was assessed using CVR and CVI. The CVR was evaluated according to the Lawshe’s formula (1975) and for all items, it was more than 0.49. Therefore, all items were approved in terms of CVR. Waltz and Baussel’s formula (1981) and the findings of Hyrkas et al. (2003) were used to calculate CVI, and for all items, it was more than 0.79; thus, all items were confirmed in terms of CVI. Gierk et al. (2018) examined the content validity qualitatively based on the opinions of mental health professionals and people who suffered directly and indirectly from mental disorders; accordingly, they considered the content validity of the questionnaire relevant and appropriate. Adeymi et al. (2018) did not examine the content validity of the questionnaire.
Internal consistency (Vronbach’s α coefficient) was used to evaluate the reliability of the questionnaire. The results showed that Cronbach’s α coefficient for the whole questionnaire was 0.86, which is considered optimal and the Cronbach’s α coefficient for discrimination (items 1-3) was 0.69 (suspicious reliability), and for both subscales of negative emotions (items 4-6) and labeling (items 7-9) was 0.74, which is the acceptable reliability. Gierk et al. (2018) reported a Cronbach’s α of 0.88 for the original German version of the questionnaire (the optimal reliability), and Adeymi et al. (2018) reported a Cronbach’s α of 0.71 for the English version of the questionnaire (the acceptable reliability). Therefore, the Iranian version has better internal consistency and reliability close to the German version compared to the English version.
The construct validity of the questionnaire was assessed using EFA and CFA. To perform EFA, first, the KMO test was performed to ensure the sample size adequacy. Then, Bartlett‘s test of sphericity was used to determine that the correlation between the variables was not zero. Given that the KMO test result was 0.854 and KMO >0.6, the sample size was suitable for factor analysis. Also, considering that Bartlett‘s test with a degree of freedom of 36 was significant at the level of <0.001, factor analysis was appropriate to identify the factor structure.
EFA using varimax rotation showed that the questionnaire had three factors: discrimination, negative emotions, and labeling that overall explained %70.1 of the total variance, and item 3 was not included in its respective component. Thus, this factor was removed from CFA. Adeymi et al. (2018) also identified the same three factors for the questionnaire and no items were removed in their research. The results of CFA confirmed the existence of three factors in the questionnaire (SRMR=0.067, P=0.17, χ2=22.4, RMSEA=0.058, FI=0.99, and TFI=0.99). 
The model fit indices showed that the model had a good fit and the standardized factor loading in all items was more than 0.4 and significant (thus, all items except item 3 had an effective and acceptable role in the formation of the relevant component.
To check the convergent validity, AVE and CR were calculated, and considering that for all three factors CR was >0.7 and AVE was >0.5 and CR was more than AVE, the convergent validity of the questionnaire was confirmed

5. Conclusion
The Iranian version of the Stig-9 questionnaire is a theory-based instrument for assessing mental health-related stigma and patients, doctors, and experts can benefit from the Stig-9 questionnaire. Evaluation in a sample of 95 military personnel with mental disorders showed that the Iranian version of the Stig-9 questionnaire has desirable psychometric properties and is an appropriate instrument for measuring perceived stigma in people with mental disorders in Iranian society.
Limitations:
This study was done using an available sampling of patients who asked for help due to MDD, BPD, adjustment disorder, GAD, OCD, panic disorder, and anxiety disorder. Although the range of disorders in the studied sample is wide, it does not represent the full range of mental disorders., like patients suffering from schizophrenia.
Another limitation is the generalization of the results to the civilian community because sampling was convenient and included military personnel.
Data for this research were taken from participants who were admitted to a psychiatric hospital for the treatment of their disorders. It can be expected that their beliefs about how society treats people with mental illness will differ from those who suffer from mental disorders but do not consult a mental health professional. Therefore, our participants may have a relatively lower perception of social devaluation, discrimination, and negative emotions toward mentally ill people.
Lack of divergent validity assessment. 
Suggestions:
Considering that the Iranian version of the Stig-9 questionnaire has been standardized in military personnel with mental disorders, it is suggested that the standardization process be carried out among civilians to better generalize the results.
The sensitivity of the Iranian version of the questionnaire to the change should be determined in intervention studies, such as anti-stigma campaigns.
The effect of gender on the Iranian version of the questionnaire should also be considered.
It is suggested that to evaluate the divergent validity of the questionnaire, in future studies, the Iranian version of the Stig-9 questionnaire be compared with other instruments that measure the stigma of mental disorders.

Ethical Considerations

Compliance with ethical guidelines
The present study was reviewed and approved by the Research Ethics Committee of the Islamic Azad University of Medical Sciences Branch (Code: IAU.TMU.REC.1399.223). All procedures were performed in accordance with relevant guidelines.
All the participants provided written informed consent and were assured that the information would remain confidential.

Funding
This research did not receive any grant from funding agencies in the public, commercial, or non-profit sectors.

Authors' contributions
Supervision: Amir Mohsen Rahnejat and Masoud Karimlou; Writing the article: Fatima Gohari.

Conflict of interest
The authors declared no conflict of interest.


References
Adeyemi, S., Abiola, T., & Solomon, K. J. (2018). Stigma and discrimination against people with mental illness: Findings from North-Central Nigeria. Malaysian Journal of Psychiatry, 24(2), 56-63. [Link]
Almanasreh, E., Moles, R., & Chen, T. F. (2019). Evaluation of methods used for estimating content validity. Research in Social and Administrative Pharmacy, 15(2), 214-221. [DOI:10.1016/j.sapharm.2018.03.066] [PMID]
Baghestani, A. R., Ahmadi, F., Tanha, A., & Meshkat, M. (2019). Bayesian critical values for Lawshe’s content validity ratio. Measurement and Evaluation in Counseling and Development, 52(1), 69-73. [DOI:10.1080/07481756.2017.1308227]
Bos, A. E., Pryor, J. B., Reeder, G. D., & Stutterheim, S. E. (2013). Stigma: Advances in theory and research. Basic and Applied Social Psychology, 35(1), 1-9. [DOI:10.1080/01973533.2012.746147]
Corrigan, P. W., & Kleinlein, P. (2005). The impact of mental illness stigma. In P. W. Corrigan (Ed.), On the stigma of mental illness: Practical strategies for research and social change (pp. 11–44). Washington, D.C: American Psychological Association. [Link]
Corrigan, P. W., Rüsch, N., & Scior, K. (2018). Adapting disclosure programs to reduce the stigma of mental illness. Psychiatric Services, 69(7), 826–828. [DOI:10.1176/appi.ps.201700478] [PMID]
Gierk, B., Löwe, B., Murray, A. M., & Kohlmann, S. (2018). Assessment of perceived mental health-related stigma: The Stigma-9 Questionnaire (STIG-9). Psychiatry Research, 270, 822-830. [DOI:10.1016/j.psychres.2018.10.026] [PMID]
Hyrkäs, K., Appelqvist-Schmidlechner, K., & Oksa, L. (2003). Validating an instrument for clinical supervision using an expert panel. International Journal of Nursing Studies, 40(6), 619-625. [DOI:10.1016/s0020-7489(03)00036-1] [PMID]
Holden, R. R. (2018). Face validity. The corsini encyclopedia of psychology, 1-2.
Holley, L. C., Stromwall, L. K., & Bashor, K. E. (2012). Reconceptualizing stigma: Toward a critical anti-oppression paradigm. Stigma Research and Action, 2(2). [Link]
Kim, T. K., & Park, J. H. (2019). More about the basic assumptions of t-test: Normality and sample size. Korean Journal of Anesthesiology, 72(4), 331–335. [DOI:10.4097/kja.d.18.00292] [PMID] [PMCID]
Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563-575. [DOI:10.1111/j.1744-6570.1975.tb01393.x]
Link, B. G., Cullen, F. T., Struening, E., Shrout, P. E., & Dohrenwend, B. P. (1989). A modified labeling theory approach to mental disorders: An empirical assessment. American sociological Review, 400-423. [DOI:10.2307/2095613]
Lucksted, A., & Drapalski, A. L. (2015). Self-stigma regarding mental illness: Definition, impact, and relationship to societal stigma. Psychiatric Rehabilitation Journal, 38(2), 99–102. [PMID]
O’Reilly, M., & Lester, J. N. (2017). History of mental health and mental illness. In: Examining mental health through social constructionism. The language of mental health (pp. 31-51).Cham: Palgrave Macmillan. [DOI:10.1007/978-3-319-60095-6_2]
Park, J. Y. (2018). Time to rethink and reverse stigma. The Journal of Medical Practice Management: MPM, 34(3), 167-171. [Link]
Stuart H. (2008). Fighting the stigma caused by mental disorders: Past perspectives, present activities, and future directions. World Psychiatry, 7(3), 185-188. [PMID]