Social skills of adolescents : Convergent validity between IHSA-Del-Prette and MESSY

This research investigates the evidence demonstrating the reliability and convergent validity between the Social Skills Inventory for Adolescents and the Matson Evaluation of Social Skills with Youngsters in two groups: the first with 136 adolescents from a public school (89 females and 47 males, with ages ranging from 12 to 17 years, 13.98±1.24 years); the second with 24 adolescents from a socio-educational program (ages ranging from 13 to 17 years, 15.64±1.18 years). The results showed low and moderate correlations between some subscales of both instruments (Matson Evaluation of Social Skills with Youngsters and Social Skills Inventory for Adolescents), evidencing convergent validity between the Social Skills Inventory for Adolescents scores and the “social skills/assertiveness” subscale of the Matson Evaluation of Social Skills with Youngsters. Comparisons between the groups indicated that the second sample had less difficulty in Civility (Social Skills Inventory for Adolescents) than the first sample (public school adolescents).

para Adolescentes e a subescala de habilidades sociais/assertividade da Matson Evaluation of Social Skills with Youngsters.
In the past few years, scientific investment in Brazil and other countries has been made to study the psychosocial and developmental conditions in adolescence.One of the key interests is socioemotional development particularly in social skills, given the typical demands at this stage of life and the understanding that social skills are learned and improved throughout life (M.Cole & Cole, 2003;A. Del Prette & Del Prette, 2001;Novak & Peláez, 2004;Z.A.P. Del Prette & Del Prette, 2005).
In adolescence, in particular, assessment of the social skills repertoire is important for detecting risk conditions and protecting against different problems, and as a cornerstone for planning and developing effective interventions (Z.A.P. Del Prette & Del Prette, 2006;Marturano, Linhares, & Loureiro, 2004;Murta, A. Del Prette, Nunes, & Del Prette, 2007).The effectiveness of programs aimed at promoting social skills in adolescence depends on the availability of instruments, procedures, and resources.This assures an accurate and reliable assessment of this repertoire, allowing for the sensible planning of these programs, for the assessment of the effects and impacts of this repertoire, and of its critical components through experimental outlines.Therefore, a more careful investment in the development and improvement of instruments with detailed psychometric qualities, in terms of both validity and reliability, is justified.
Reliability refers to indicators of accuracy or fidelity, generally investigated through test-retest procedures (temporal stability), correlations between parallel forms of the test itself, and the internal consistency of items.Validity is expressed by two main concepts: first, the representation of the construct under evaluation in the processes of resolution or response to test items; and, second, the relation between the scores of a test and other variables measuring the same or related constructs (Kamphaus & Frick, 2002).Evidences of validity can be found in several ways, namely, by judging the content of the item, analyzing factorial structure, or comparison with other instruments.Convergent validity follows from this reasoning.It is based on empirical evidences of a nomological span of relations between directly or inversely correlated constructs.The evidences of validity and reliability represent definite cumulative results, and it can be stated that the psychometric value of an instrument is commonly reinforced or lessened according to new psychometric studies.This justifies continuous investment in this area (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1999).
Social skills assessment instruments for adolescents are still scarce in Brazil.However, two self-report instruments recently developed and adapted to assess the social skills repertoire and concurrent problematic behaviors show initial indicators of reliability and validity, and are currently under additional investigation.The first instrument is MESSY (Matson Evaluation of Social Skills with Youngsters) (Matson, Rotatori, & Helsen, 1983), developed in the United States and adapted to Brazil by Teodoro, Käppler, Rodrigues, Freitas and Haase (2005).MESSY provides scores on four important subscales for the assessment of social behavior in adolescents: ( 1 (2) Self-control; (3) Civility; (4) Assertiveness; (5) Affective approach; and (6) Social resourcefulness.
The convergent validity of the two instruments can be investigated in terms of the network of potential relationships between the indicators they provide.Based on the literature regarding the social skills mentioned previously, positive correlations can be expected among the indicators of the good social skills repertoire (high frequency and low difficulty on IHSA-Del-Prette, and scores of Social Skills/Assertiveness on MESSY), as well as negative correlations between these results and the three scales that assess behavioral problems on MESSY.Given the previous considerations, the purpose of this study was to investigate aspects of validity and reliability of the Social Skills Inventory for Adolescents (IHSA-Del-Prette) grouped into three areas of analytical investigation: (1) assessment of age and gender differences on IHSA-Del-Prette scores and internal consistency of items for the sample studied; (2) comparison between IHSA-Del-Prette frequency scores and MESSY scores validated for Brazil; and (3) comparison between the scores of two samples on both instruments, with one of the samples composed by adolescents in vulnerable situations.

Participants
Participants were assigned to two groups.The first group was composed of 136 adolescents from a public teaching institution in the state of Rio Grande do Sul, Brazil [89 females (65.40%) and 47 males (34.60%) between the ages of 12 and 17 years (M=13.98,SD=1.24)].The second group was composed of 24 adolescents participating in a Social/Educational Program (SEP) in an average-size city in the state of São Paulo.Adolescents were aged between 13 and 17 years (M=15.64,SD=1.18); 17 were boys (72%) and 6 were girls (24%).One adolescent (4%) did not answer the questionnaire.The composition of the sample was intentional.

Instruments
Social Skills Inventory for Adolescents (IHSA-Del-Prette).The Social Skills Inventory for Adolescents (IHSA-Del-Prette) (Z.A.P. Del Prette & Del Prette, 2009) is a self-report instrument to assess social skills in adolescents.It is referenced to the standard percentiles, and was investigated and approved by the Federal Council of Psychology.The inventory is composed of 38 items, taking into account relationship skills with different interlocutors (affective/sexual partner, parents and siblings, acquaintances, friends, people with authority, strangers, or unspecified) required for public (school, work, leisure, consumption), private (family and intimate), or unspecified contexts.In each item, the adolescent is requested to estimate (a) his/her level of difficulty in presenting the reaction indicated in the specific item; and (b) the frequency with which he/she presents that reaction.The answers are marked on a Likert-like scale, providing general scores of difficulty and frequency.The IHSA-Del-Prette also provides scores on six subscales: (F1) Empathy; (F2) Self-control; (F3) Civility; (F4) Assertiveness; (F5) Affective Approach; and (F6) Social Adroitness.Analysis according to Cronbach's Alpha shows a score of 0.90 for the overall scale, with rates ranging from 0.51 to 0.87 for the subscales.The IHSA-Del-Prette comes in a box containing a printed manual, as well as application and scoring cards.Online scoring is also available.
Matson Evaluation of Social Skills with Youngsters.MESSY is a self-assessment scale developed by Matson et al. (1983).It contains 62 items, completed on a Likert scale.In its original version, items were grouped into six factors: Appropriate Social Skill, Inappropriate Assertiveness, Impulsive/Recalcitrant, Overconfidence, Jealousy/ Withdrawal and Miscellaneous Items that include the other non-classified items.MESSY was translated into Portuguese, and its psychometric properties were investigated by Teodoro et al. (2005).Its Brazilian version contains the same 62 items divided into four factors: (1) Aggressive/ Antisocial Behavior, (2) Social Skills/Assertiveness, (3) Conceit/Haughtiness and (4) Loneliness/Social Anxiety, analyzed according to exploratory and confirmatory factor analysis.The internal consistency rates were 0.87, 0.84, 0.74 and 0.47, respectively.

Procedures
The locations studied (schools and SEP) were contacted by the directors.After authorization, the goals of the study were explained to the participants.Parents were requested to sign the Informed Consent Form.The project was approved by the Ethics Committee (CEP/Universidade Federal de São Carlos nº 206/2005).

Data treatment
Comparisons of age and gender for the social skills scores, as well as comparisons between both instruments studied and the calculation of internal consistency were carried out separately for both groups.Owing to the study sample size, parametric tests (T-Test and Pearson's Correlation) were employed for the public school group, and non-parametric tests (Mann-Whitney and Spearman correlation) were employed for SEP participants.
Internal consistency was assessed by Cronbach's Alpha.The associations between IHSA-Del-Prette and MESSY were calculated with the Pearson correlation coefficient.Comparisons between vulnerable and non-vulnerable groups were made with Univariate Variance Analyses.
In the SEP group, girls scored significantly lower regarding frequency on the IHSA-Del-Prette self-control subscale (z=2.14, p<0.05) and higher on Civility (z=2.70,p<0.05).In terms of difficulty, boys scored significantly higher on Social Adroitness (z=2.09,p<0.05).Gender comparisons on MESSY showed that girls scored higher than boys on Assertiveness (z=2.46,p<0.05).
Pearson correlations showed a significant negative association between age and IHSA-Del-Prette factors for the public school group (r=-0.30), in terms of difficulty regarding Social Adroitness.No significant associations were found between age and MESSY scale scores.In regard to the SEP group, a positive association was observed between age and self-control frequency scores (r=0.46) on IHSA-Del-Prette.

Internal consistency analyses
Internal consistency analyses (Cronbach's Alpha) of the skills on each of the IHSA-Del-Prette subscales were calculated for the total sample.The rates found for frequency scores were 0.80 for Empathy, 0.79 for Self-control, 0.78 for Civility, 0.70 for Assertiveness, 0.63 for Affective Approach and 0.64 for Social Adroitness.The overall scale of behaviors showed an Alpha of 0.91.For the difficulty scores, an Alpha of 0.90 was observed for Empathy, 0.79 for Self-control, 0.90 for Civility, 0.75 for Assertiveness, 0.67 for Affective Approach and 0.54 for Social Adroitness.An Alpha of 0.92 was observed for the overall score for difficulty.The internal consistency rates regarding the MESSY subscales were 0.86 for Aggressive/Antisocial Behavior and Social Skills/Assertiveness, 0.67 for Conceit/Haughtiness, and 0.40 for Loneliness/Social Anxiety.

Convergent Validity between IHSA-Del-Prette and MESSY
Data taken from IHSA-Del-Prette and MESSY, for adolescents from the public school, were investigated according to Pearson correlations.Table 1 shows the scores for the frequency indicator on both IHSA-Del-Prette and MESSY.

Comparison between groups
Comparisons between the group of adolescents from the public school and that of adolescents from the SEP were made with Univariate Variance Analyses controlled by gender.No differences between the groups were found in the frequency scores investigated by IHSA-Del-Prette.However, adolescents from the SEP group showed less difficulty on Civility than those from the public school (F=5.72,p<0.01).No interaction was found between the group of origin and the gender of the participant.
For a more in-depth assessment of the differences between groups regarding Civility, IHSA-Del-Prette items were compared.Results are shown in Table 2.
The group of adolescents from the public school and that of adolescents from the SEP showed significant differences in four of six items on Civility.Adolescents from the public school reported more difficulty in four of the items: D2 [I greet people when I arrive somewhere (doctor's office, relative's

Discussion
Considering the first goal (comparisons of age and gender), our results show that approximately one third of the analyses of gender difference assessment (11 out of 36) were significant, suggesting isolated gender-related differences on social skills scores.However, we must point out that the results obtained from the normative, broader and more diversified sample of adolescents (Z.A.P. Del Prette & Del Prette, 2009) were more robust in terms of gender differences, indicating different norms for each gender.Moreover, the gender difference analyses for the SEP group included only a small number of girls.
In terms of age, there were only two significant correlations from a total of 36.This indicates that the results were probably incidentalthat is, the number of significant correlations is very close to the percentage of error (5 out of 100).Therefore, we can conclude that age played no role in regard to the age gap studied.This is unexpected, since the normative sample showed different results for the age gaps of 12-13, 14-15 and 16-17, thus implying different norms for each age gap.The two earlier results suggest that the study sample is very different from the IHSA-Del-Prette normative sample, in terms of age and gender impact on social skills scores.The difference in results could be attributed to the specificity of the sample, i.e., taken from a small city in the state of Rio Grande do Sul, Brazil.
Data from the Alpha scores suggest that there is an internal consistency for the global scale of both instruments, with satisfactory indicators for both IHSA-Del-Prette and MESSY (Pasquali, 2003).Four of the six IHSA-Del-Prette subscales and one of the MESSY subscales showed above average (greater than or equal to 0.70) internal consistency.In addition, the consistency of items assessing difficulty was greater than that of items assessing frequency.This suggests the greater reliability of this type of measure.Results show reliability of results for the overall score of both inventories and most subscales, especially those related to social skills.
The IHSA-Del-Prette (frequency) versus MESSY results related to convergent validity were as expected in terms of positive correlations between the social skills indicators of both instruments, that is, between IHSA-Del-Prette subscales and the MESSY Social Skills/Assertiveness subscale.Although Table 1 states seven significant correlations, only four of these showed magnitudes above 0.40.These were observed for the MESSY social skills/assertiveness subscale and: the overall IHSA-Del-Prette (0.51); the subscales of Empathy,Civility,and Social Adroitness (r=0.59,r=0.52,and r=0.44,respectively).These results evidence convergent validity between desired social behaviors measured by both scales, in agreement with data found in the literature (Hinshaw, 1992;Lane et al., 2002 The results for the three additional MESSY factors, indicating behavioral problems, were unexpected.Considering the protective factor of the social skills and the risk factor of deficits within this repertoire of behavioral problems (Walker & Severson, 2002;Crews, Bender, Gresham, Kern, Vanderwood, & Cook, 2007), as well as the concurrent nature of both repertoires (Gresham, Van & Cook, 2006;Patterson & Yoerger, 2002), it could be expected that there would also be convergent validity in terms of significant negative correlations between IHSA-Del-Prette scores and the three MESSY behavioral problem subscales.Correlations were negative, as expected; however, they were very low or non-significant overall, except for Aggressive/Antisocial Behavior, which showed significant negative correlations between 0.21 and 0.35, with several IHSA-Del-Prette indicators (overall score, Empathy, Self-control and Civility).
It should be noted that even in the study for MESSY validation in Brazil, the three factors related to behavioral problems showed very low correlation with the social skills/assertiveness indicator on the same scale.By assessing the content of the items regarding the three MESSY factors (such as I behave as if I was better than others, I want to lead and I like to be alone), we can see that most of these factors do not necessarily imply social skill deficits, but rather the inappropriate use of such a repertoire, according to criteria of social competence (A.Del Prette & Del Prette, 2001).In this sense, our data endorse the idea that a good repertoire of social skills is a necessary asset, but not enough for a socially competent performance (A. Del Prette & Del Prette, 2001;Z.A.P. Del Prette & Del Prette, 2005).This might explain the low correlation rates between positive and negative indicators of both scales, both in this study and in the study to validate MESSY in Brazil.In other words, this study did not provide evidence of convergent validity for the negative indicators of MESSY; rather, the term divergent validity might apply, because the items on these subscales investigate aspects that are different, but not necessarily the opposite of social skills.
The comparison of both samples of this study suggests that the difference between adolescents from SEP and those from the public school was not substantial; therefore, the SEP adolescents do not constitute a risk sample.This conclusion is also surprising, because the SEP adolescent sample is different from another sample from the same institution, analyzed in a different study (Del Prette et al., 2012).The SEP sample showed a prevailing deficit on social skill scores and indicators of psychological disorders.The analysis of items showing differences between these two samples suggests that, quite incidentally, the SEP sample in the present study constituted a subsample that does not represent the pattern of adolescents in this program.We could also assume that our small sample size (N=25) did not take into account the diversity of individuals included in the other study (N=50) or in the SEP, which could explain our results.A third plausible hypothesis is that, because of the characteristics of the program itself, adolescents from the SEP could represent a subgroup more attentive and motivated to meet the demands of the institution in terms of social performance, and could, therefore, present a pattern compatible with the reference sample.It is important to observe that the socioeconomic conditions of participants were not considered.Together with cultural differences, socioeconomic conditions may influence comparative results.Therefore, the comparison between adolescents from the SEP and those from regular schools is still open for future investigation.We could further infer that a broader sample of the first group is needed.

IHSA
Social Skills with YoungstersNote: * p<0.05; ** p<0.01; *** p<0.001;IHSA-Del-Prette: Social Skills Inventory for Adolescents.home, for example)], D3 (I thank someone who does me a favor or a good turn), D4 (I say goodbye to everyone when I leave a place) and D9 (I thank people who compliment me).

Table 1
Pearson's correlation between Frequency Scores on the Social Skills Inventory for Adolescents and Scores on Matson Evaluation of Social Skills with Youngsters for the Public School Sample (n=136).São Carlos (SP), Brazil

Table 2
Descriptive data (average, standard deviation) of difficulty scores for the IHSA-Del-Prette social skills items regarding the Civility subscale (overall variation from 0-2) and inferential data (t value and significance) for differences between public school and Social/Educational Program (SEP) adolescents.São Carlos (SP), Brazil