Special Issue: Psychometrics
In this Issue
Special Issue: Psychometrics
HIV-Related Scales Psychometrically Validated for Rural African-American Women
Emma (E.J.) Brown, PhD, APRN-BC, FAAN
Huey-Shys Chen, PhD, RN
Culturally relevant measures to evaluate the effectiveness of HIV interventions informed by rural African-American women are warranted. Themes associated with selecting HIV negative sexual partners emerged during formative research with this subgroup.4 Specific measures for this concept have not been developed or psychometrically tested. The reluctance of rural African Americans to complete surveys also emerged.3 The study aim was to develop or modify and psychometrically evaluate brief HIV-related scales for African-American women. A 20-page HIV-related instrument was administered to 200 African-American women from rural communities and small cities. Presented are psychometrics of two scales, the decision-making regarding partner selection (DMRPS) scale and a brief attitude toward condom use (ATCU). The brief DMRPS and ATCU scales demonstrated satisfactory construct validity and internal consistency reliability, providing evidence to support their use in HIV prevention intervention studies with rural African-American women.
Keywords: African-American women, rural, HIV, psychometric testing, scale development
HIV-Related Scales Psychometrically Validated for Rural African-American Women
The development of reliable and valid measures, which are gender and regional specific, is necessary for use with HIV prevention interventions targeting Southern rural African American women. Operationalization of partner selection and attitude toward condom use are two such measures. Some researchers suggest that scant findings about the factors associated with women’ attitudes about their partners’ use of male condoms have been written.28 Therefore, formative and quasi-experimental research that assists in the development and testing of brief scales, which advance understanding of the barriers or facilitators associated with HIV risk and risk-reduction behavior among rural African American women is warranted.
HIV/AIDS and the Need for Related Prevention Interventions
The southern region of the United States had the highest rates of chlamydia, gonorrhea, and syphilis in 2001 according to the Centers for Disease Control and Prevention.7 Increasing sexually transmitted infection (STI) rates are especially significant since they facilitate HIV transmission at least two- to five-fold.26 Additionally, African Americans comprise 75% of the AIDS cases in the southeastern United States with HIV rates of infection among African Americans increasing disproportionately to other racial/ethnic groups in the region.15 Nationwide, HIV/AIDS was the leading cause of death among African American women ages 25 to 44.8
To date, limited published findings of prevention intervention research among southern rural African American women exist such as the one published by Wingood, Hunter-Gamble, and DiClemente.31 Gender specific HIV prevention intervention studies are needed in rural areas as rural African American women, and especially southern rural African America women, tend to hold more firmly to traditional gender roles -- i.e. deferring to males, exhibiting passive behavior.2 Internalization of traditional gender roles and resulting behavior may explain in part the existence of factors associated with HIV risk, such as not discussing past sexual behavior or condom use with partners among rural African American women.
Scant are HIV-related instruments normed on rural samples as well as brief instrument, which may enhance reliable data collection with rural and minority populations.3,14 Previous process evaluation of southern rural African Americans revealed that including data collection instruments was a barrier to recruiting participants in HIV prevention intervention programs3 and the use of lengthy data collection instrument was negatively received by participants of a research study.14 Participants of the latter study complained that completing two surveys (pre- and post-intervention) was unnecessary because both included the same items.
Use of male condoms in heterosexual relationships decreases as duration of relationship increases or perception of relationship commitment strengths.17,19 These findings have implications for initial partner selection in HIV risk reduction. Several themes (e.g. use of subjective rather than objective criteria in selecting sexually transmitted infection (STI) and HIV-negative sexual partners) emerged during recent formative studies among southern rural African American women.4-6 Again studies with rural populations are limited. In a study among urban African American women29 findings revealed that perception of partners’ risks were inconsistent, either underestimated or overestimated, with partners’ self-reported risks. In a study conducted by Sobo25 urban women indicated they perceived that they were in monogamist relationships, which posed little risk of them acquiring HIV infection. Participants’ in that study simultaneously perceived that other women were at risk of acquiring HIV infection due to their partners’ sexual behavior.25 In other words, the sexual partners of other women were non-monogamous while their partners were monogamous. In all these studies, women use subjective rather than objective criteria in their assessment.
Attitudes Toward Condom Use
Attitude toward condom use among rural African American women have been conducted much less extensively than among urban African American women.1,17-19,27 The major findings of studies among urban women indicate that younger age, higher self-esteem, belief in conspiracy theory, history of childhood neglect and casual partner type were associated with positive condom use attitudes. In contrast, older age, social norms not supportive of condoms use, commitment and long-term relationship were associated with negative condom use attitude. Studies reporting findings among rural African American women were not located, which indicates a major gap in the research literature regarding rural African American women’ attitudes toward condom use and further suggest the need for additional research about condom use attitude among rural African American women.
The development of reliable and valid measures, which are culturally-, gender-, and regionally- specific is a vital component in the process of evaluating the efficacy and effectiveness of culturally competent HIV prevention interventions. A strategy recommended to achieve this objective is the involvement of individuals from the targeted population in the research process (participatory action research) during the early developmental stages.30 The purpose of this paper is to present the psychometric findings of a brief Decision-making Regarding Partner Selection (DMRPS) scale and an Attitude Toward Condom Use (ATCU) scale for African American women from rural areas and small cities.
A survey research design was used in this study. The institutional review board of the University of Central Florida approved the study.
From 200 to 300 subjects are needed to evaluate the reliability of newly developed instruments.12 For principal component factor analysis, a recommended ratio of subjects to items is 10:1.11 The DMRPS has 13 items; requiring at least 130 participants. Since this was the scale with the largest number of items, a sample size of 200 was deemed appropriate.
For the initial questionnaire completion 34 key informants who met the following inclusion criteria: African American, female, ages 18 to 39, and either attended an STI clinic or resided in neighborhoods with high drug activity, were recruited because of their potential HIV risk to participate in the study through flyers posted an STI clinical waiting areas and on mailboxes within housing developments. Key informants were instructed to call the research staff to set up data collection appointments for their network to complete the questionnaire. Research staff informed key informants of the inclusion criteria and the research protocol, including the time commitment and subject reimbursement during these calls. Once recruited, the 34 key informants then recruited 166 other women from their social network who met the inclusion criteria for a total sample of 200 participants.
According to Fleiss,20 subjects are sufficient to estimate the reliability of a measure.13 Therefore, forty (20%) of the 200 participants who completed the initial questionnaire were contacted and recruited by the key informant to retake the questionnaire to evaluate its test-retest reliability. The second data collection occurred two weeks after they completed the initial questionnaire.
Measures with Operational Definitions
Research participants who met the inclusion criteria completed a paper and pencil instrument, which included the following sections (demographics-15 items; history-8 items; HIV-related behavior-20 items; drug/alcohol behavior-15 items; and social network-10 items) and multiple scales; DMRPS-13 items; attitude toward women-1 item; ATCU-7 items; and HIV-related knowledge-19 items. The Flesch-Kincaid grade level for the instrument was 6.1. This manuscript reports the psychometric results of the DMRPS and the ATCU scales only.
The entire DMRPS was constructed based upon findings from formative studies, which focused on rural and small city participants’ perception of HIV transmission, and sex and drug-use behaviors. Three studies, two focus groups5,6 and one in-depth individual interview4 were conducted. The conceptual definition of DMRPS is one’s conscious decision making regarding the selection of sexual partners perceived as less likely to be infected with HIV. The authors developed a multidimensional item pool, but did not know how many components would emerge. The content validity of the scale was evaluated using the layperson approach, that is, by evaluations from women from the targeted group not experts. Participants of the focus groups previously referenced were asked to assess the wording and clarity of each item on the scale. The wording suggested by the focus group participants was used without modification for this scale. Therefore, contractions such as “don’t” are used since they may have regional significance.
The lead sentence on the DMRPS scale was “What are the things that you look for in a sexual partner that you think indicate he does not have HIV infection? (Please circle how much you disagree or agree with each statement on a scale of 0 to 10 with 0 being disagree 100% and 10 being agree 100%.)” The items were: 1) if they’ve been taught well by their parents, 2) if they are not skinny, 3) if their body is clean, 4) if they dress well, 5) if they go to church, 6) if they drive a nice car, 7) if they don’t sleep around, 8) if they want to use condoms when they first meet you, 9) if they don’t use drugs, 10) if they have a regular job, 11) if you were friends with them beforehand, 12) if you know who they hang out with, and 13) if you know their family.
The conceptual definition of ATCU is one’s attitude toward using condoms, whether positive or negative. The authors expected this scale would be bi-dimensional, including both positive and negative attitudes. Content validity for the ATCU was based on a review of the literature and focus group data with African American women who resided in rural areas and small cities. Three of the original items on this scale emerged during focus group studies.5,6 The remaining four items, normed on urban African American and Hispanic women, were adapted from Hammer, Brenner, Rhodes, and Shields16 and used with permission.
The lead sentence of the Attitude Toward Condom Use scale was “please indicate how much you agree or disagree with the following statements. If you completely disagree, circle 0. If you completely agree, circle 10.” The original seven items were: 1) it is easy to use condoms, 2) it is affordable to use condoms, 3) I will only have sex with a condom, 4) using a condom during intercourse is a turnoff, 5) I worry about whether potential sex partners will enjoy sex using a condom, 6) condoms are the best way to protect myself from HIV, and 7) condoms are the best way to protect myself against other sexually transmitted infections.
Data collection for the initial instrument occurred over a two-week period while data collection for the test retest component was about three days. The data collection sites were key informants’ apartments and/or outdoor settings where the key informants dwelled. A methodological paper detailing the data collection method of this study14 has been published elsewhere. Upon arriving at the data collection site and verifying who the participants were, the PI reviewed the purpose of the study, the procedure for completing the instrument, and verbal consent information with the participants. Time was allocated to respond to participants’ queries and receive their verbal consent before distributing the instrument. Written informed consents were not collected since the omission of names and other identifiers were found to enhance participation in past studies with similar populations.6
A unique identifier was included on the first page of the instrument and was matched with contact information, which was collected and stored in separate file cabinets. The contact information was used to contact forty of the original participants to complete the instrument a second time to evaluate the test-retest reliability. The duration of time for the completion of the instrument was between 45-70 minutes. Participants received $20 each time they participated in the study.
Data were analyzed using the Statistical Package for Social Sciences (SPSS) version 12.0. The data file was examined for accuracy, missing data, and outliers. Frequencies were computed to identify out of range values. Pairwise procedures were applied in SPSS to handle missing data; that is, variables with missing values were excluded pairwise from the analysis.
In order to perform more mathematical operations and attain more precision in measurement, it is appropriate to sum the individual item score on each item of a Likert type scale to produce an interval-level total scale variable.20-22 Therefore, while Cronbach's alpha was computed on each scale using all individual items as data. Pearson product-moment correlation coefficients using total scale scores were computed to assess the 2-week test-retest reliability. The test-retest procedures were conducted before the factor analysis procedures.
Principal components factor analyses were computed to study the underlying structure and estimate the construct validity of the DMRPS and the ATCU scales. Exploratory factor analysis procedures with varimax rotation were conducted to find the identifiable attributes of the constructs since they were newly developed or in the early stage of development as suggested by Dixon.10 Pedhazur and Schmelkin22 suggest that only the items with factor loadings exceeding 0.30 or 0.40 are meaningful in applications of factor analysis. In this study, items were retained only if the factor loading was above 0.40. In addition, only components with eigen values greater than 1.00 were retained for interpretation. The principal component analysis was conducted using casewise deletion; 197 out of the 200 cases were used in the procedure.
Participants ranged in age from 18 to 39 years with a mean age of 28.1 years and reported the use of alcohol and other drugs. The 200 participants were drawn from 34 social networks of which 26% were illicit drug using networks and 30% were from very rural areas. See Table 1 for detailed individual level demographics.
Conscious Decision-making Regarding Choice of Less Risky Partner (DMRPS)
Construct validity. After principal component analysis was calculated, three items were deleted from the DMRPS scale because they loaded on more than one component.11 These three items were: 1) if they had been taught well by their parents, 2) if they are not skinny, and 3) if their body is clean. The revised DMRPS instrument with 10 items that loaded on three components accounted for 77.3% of the total variance on the DMRPS (see Table 2). Item-subtotal correlations were conducted to explore the validity of the revised DMRPS components. The items for each of the three components were summed to yield a subscale score; then each item was correlated with the subscale score for the component on which it loaded to estimate item to subtotal correlation.
The first component accounted for 29.8% of the variance and included four items. Theoretically, these items represented women’s tendency to select less risky partners based on externalities; therefore, this component was labeled as external entity. The item loadings for these four items were all above 0.69, with a range of 0.69 to 0.84.
The second component, labeled as knowledge of family/social network, accounted for 24.9% of the variance and comprised three items. These items represented women’s tendency to select less risky partners based on their knowledge of the partner’s family and friends. The item loading range was from 0.76 to 0.80.
The third component, risk-reduction behavior, accounted for 22.4% of the variance and had three items, which represented women’s tendency to select less risky partners based on the partners’ use of risk-reduction behavior. The smallest item loading for these three items was 0.57, with a range of 0.57 to 0.65.
The item-subtotal correlations ranged from 0.70 to 0.80 for external entity, 0.69 to 0.86 for knowledge of family/social network, and 0.57 to 0.65 for risk-reduction behavior. Reliability of the revised DMRPS was reassessed after dropping three items. Cronbach alpha coefficients for these three subscales were 0.89, 0.89, and 0.78, respectively. The study results indicated that three subscales of the DMRPS had acceptable construct validity and reliability.11,21 In addition, the 2-week test-retest reliability of the revised DMRPS was assessed by the Pearson product-moment correlation coefficient with a value of 0.74, which demonstrated the DMRPS had acceptable consistency over time.9,21
Attitudes Toward Condom Use (ATCU)
Reliability. Reliability was assessed for the ATCU using Cronbach’s alpha. The original scale, which included three of the seven items with low item-total correlation coefficients, had a Cronbach alpha of 0.64. Nunnally and Bernstein21 recommend retaining items with item-total correlation coefficients above 0.30 so three items were dropped from the original ATCU. These three items were: 1) it is easy to use condoms, 2) it is affordable to use condoms, and 3) I will have sex only if I use a condom. The Cronbach coefficient alpha for the revised 4-item ATCU significantly increased to 0.78, which indicated an acceptable level of internal consistency.21 The 2-week test-retest reliability of the ATCU was assessed by the Pearson product-moment correlation coefficient with a value of 0.75 after dropping three items, which demonstrated the ATCU had acceptable consistency over time.9,21
The first component including two items representing women’s attitudes toward condom use related to the benefits of condom use accounted for 49.8% of the variance. This component was labeled “advantage of condom use”. The item loadings for these two items were both above 0.90. The second component including two items representing women’s negative attitudes toward using condoms with their primary partners accounted for 41.1% of the variance, and was labeled “disadvantage of condom use”. Both items loaded at 0.88 or above. The correlation coefficient between the two components was -0.58.
Item-subtotal correlations were conducted to further explore the validity of the ATCU component. The two items for each component was first summed to yield a subscale score. Then each item was correlated with the subscale score for the component on which it loaded. The item-subtotal correlations were 0.63 or above for the advantage and disadvantage subscales. Cronbach alpha coefficients for the advantage and disadvantage subscales were 0.96, and 0.78, respectively. The two subscales had acceptable construct validity as each item highly correlated with the composite subscale9,12 and also had acceptable levels of internal consistency.11,21
The study aim, to develop and test the validity and reliability of two brief HIV-related prevention scales for rural areas and small-cities African American women, was achieved. The conscious decision-making regarding the choice of less risky partner (DMRPS) scale demonstrated satisfactory construct validity and internal consistency reliability in a population of rural and small city African American women. The amount of variance explained by the most objective criteria, the women’ tendency to select less risky partners based on the partners’ use of risk reduction behavior, was 22.4%, in contrast to the subjective criteria of the women assessing partners’ HIV risk based on the women’ knowledge of the partners’ family and friends (24.9%) and externalities (29.7%). The variance explained by subjective decision-making criteria were greater than that explained by the objective decision making criteria, 54.6 and 22.4%, respectively. HIV prevention interventions must aim to decrease women use of physical attributes or appearance as the bases on which to determine their partners’ HIV risk.
More objective criterion such as partner’s use of condoms during the early stage of the relationship, partner’s lack of drug use, and knowledge of partner’s past sexual history are more likely than the other partner selection criteria to result in accurate appraisal of one’s sexual partner’s HIV risk--although not valid in determining partners HIV serostatus. Partner selection criteria such as: 1) being friends with partners before engaging in a sexual relationship, 2) knowledge of who partners hang out with, and 3) knowledge of partners family although less valid than partner selection criteria based on partners’ risk or risk reduction behavior appear to tap into the “southern rural cultural norms.” Southern cultural norms seem to equate knowledge of individuals with what is known about individuals’ family and to a lesser degree their friends. The latter partner selection criteria may, in turn, be more valid than criteria based on external characteristics such as how partners dress, if they go to church, if they drive a nice car, or if they have a regular job.
Although several limitations (convenience sampling schema, modest sample size, fewer adolescents and young women, etceteras) were inherent in the study design the development and testing of these scales adds to the existing body of knowledge about decision making about partners risk as well as about valid and reliable brief scales. The aim of developing brief scales was achieved. Components that facilitate more realistic appraisal of partners HIV risk, which should be emphasized in HIV prevention interventions and distinguished from the component that facilitates erroneous appraisal of partners risk, which should be corrected, were identified in the DMRPS. The DMRPS appears to have the potential to serve as a measure to assess the type of information and kind of activities needed in HIV prevention interventions as well serving as a useful outcome measure to evaluate the efficacy of such interventions.
The ATCU scale also demonstrated satisfactory construct validity and internal consistency reliability in this sample of rural and small city African American women. This result is helpful because while other attitude toward condom use scales exists the modified scale evaluated in this study is the only 4-item brief scale that has been evaluated to be valid and reliable. This scale was bi-dimensional and captured the negative attitudes toward condom use as well as the benefits of using condoms. Both dimensions suggest avenues for intervening with the targeted population.
These findings provide evidence of the validity and reliability of the DMRPS and ATCU scales. The process by which the items were generated and selected, by the women themselves, adds to the cultural relevance and gender and regional specificity of the scales. The findings support the continuous use of these scales in HIV prevention intervention studies and programs targeting rural and small city African American women, although the convenience sample is not representative of all African American women from rural areas and small cities.
Table 1. Sample Demographics: Individual level Demographics (N=200)
Note: *Four participants did not answer the level of education item. **Participants indicated all drugs used, so the number exceeds 200.
Table 2. DMRPS Scale: Final Results of the Varimax Rotation and Item Analysis
Note: DMRPS = Decision-making Regarding Partner Selection. The lead sentence on this scale was what are the things that you look for in a sexual partner that you think indicate he does not have HIV infection?
Table 3. ATCU Scale: Final Results of the Varimax Rotation and Item Analysis
Note: ATCU = Attitudes toward condom use.