Revista Brasileira de Psiquiatria ISSN print 1516-4446
ISSN on-line 1809-452X
JCR IF 2018: 2.440
Fully open access
No submission fees
No publication charges

Braz J Psychiatry Ahead of Print


Bullying among adolescents: are the victims also perpetrators?

Andressa Reisen1; Maria C. Viana1,2; Edson T. dos Santos-Neto1,2

1. Programa de Pós-Graduação em Saúde Coletiva, Universidade Federal do Espírito Santo (UFES), Vitória, ES, Brazil; https://orcid.org/0000-0003-3916-462X
2. Departamento de Medicina Social, UFES, Vitória, ES, Brazil

Andressa Reisen
Av. Vitória, 1729, Jucutuquara
CEP 29040-780, Vitória, ES, Brazil
E-mail: areisen@hotmail.com

Received August 16, 2018
Accepted after revision January 8, 2019

Descriptors: Adolescent; bullying; adolescent behavior; schools; violence.
Abstract
OBJECTIVE: To analyze factors associated with the prevalence of maltreatment and bullying and to identify types of involvement (verbal, physical, social, sexual, cyberbullying) among high school students aged 15 to 19 years.
METHODS: A cross-sectional, school-based epidemiological survey was performed. The sample included 2,293 adolescents from public and private schools in the Greater Vitoria area (state of Espírito Santo, Brazil). A modified version of the Brazilian Portuguese Olweus Bully/Victim Questionnaire was used.
RESULTS: Among maltreatment behaviors, 43.3% of adolescents reported having been victims vs. 40.4% reporting to be aggressors. Among bullying behaviors, 41% reported victimization and 29.1% aggression. The most frequent types of bullying were verbal (victim = 33.8%, bully = 23.1%), social (victim = 21.8%, bully = 16.9%), and physical bullying (victim = 15.1%, bully = 8.7%). Of those reporting to be victims, 37.5% stated that they did not react as frequently as they were attacked. Almost half of the students (50.9%) identified themselves as victims, without practicing any type of aggression against another schoolmate. School network (public or private) and gender were significantly associated with victimization and aggression behaviors.
CONCLUSION: The adolescents identified as victims did not generally attack other students, i.e., did not identify themselves as perpetrators. The high prevalence of maltreatment and bullying detected in this study, especially the verbal, social, and physical types, underscores the need for interventions addressing bullying in schools.

INTRODUCTION

Violence is a growing problem around the world, and its signs are also felt in the school environment. In schools, bullying is one of its most visible forms of violence,1-3 associated with deleterious consequences for the physical and mental health of the individuals involved,4,5 possibly hindering intellectual development and compromising the social potential of children and adolescents.1,6

Bullying involves aggressive attitudes by one or more students against another, characterized by intentionality, repetition, and power asymmetry.1-4 It involves different forms of behaviors, according to which students may be classified as aggressors, victims, victim-aggressors, and witnesses, based on their attitude.1,2 Bullying has also been categorized according to the type of event and forms of manifestation: physical, verbal, social, sexual, and cyberbullying.4,5,7,8 When these aggressive behaviors occur less frequently, they are called maltreatment.4

Scientific studies carried out in Brazil have revealed a broad variation in the prevalence of bullying behaviors. A pioneering study including more than 5,000 students in the city of Rio de Janeiro (state of Rio de Janeiro) performed in 2002 and 2003 by the Brazilian Multiprofessional Association for the Protection of Children and Adolescents (Associação Brasileira Multiprofissional de Proteção à Infância e à Adolescência [ABRAPIA]) revealed that 16.9% of the students reported having been victims of bullying, while 12.7% were characterized as bullying abusers.1 Research carried out in the city of Esteio, state of Rio Grande do Sul, in 2009 showed that among the evaluated students, 56.9% were victims of bullying and 38.5% were perpetrators of bullying.9 These results agree the high prevalence identified in a study conducted in the Rio Grande do Sul state capital Porto Alegre in the same year, which showed 67.5 and 54.7% for victims and aggressors respectively.2 The results of a study carried out in the city of Olinda, state of Pernambuco, in 2012 indicated that almost half of adolescents aged 13 to 19 years were victims of bullying, while approximately one-third showed aggressive behavior.5

Bullying behaviors have attracted growing attention and interest from researchers and society over the past years. Understanding bullying in its various aspects is essential to ensure the optimal biopsychosocial development of children and adolescents, by providing them with a healthy, safe, and violence-free environment.1,5,6,10

According to the World Health Organization (WHO),11 adolescence encompasses the second decade of life (10 to 19 years of age), and can be divided into two periods: pre-adolescence, 10 to 14 years of age; and adolescence, 15 to 19 years of age (late adolescence). It is a period characterized by profound biological, cognitive, emotional, and social changes, associated with a greater occurrence of bullying.2,12,13 However, studies that address bullying in late adolescence are less frequent in Brazil.1,10,13,14 In this sense, investigations addressing bullying in late adolescence are required for the understanding of determinants and constraints, in order to produce effective prevention interventions.

This study aims to identify the factors associated with the prevalence of maltreatment and bullying victimization and aggression, as well as to describe the types of involvement (verbal, physical, social, sexual, cyberbullying) in late adolescence (15 to 19 years), in students enrolled in public and private high schools in the Greater Vitória Area (Região Metropolitana da Grande Vitória - state of Espírito Santo [RMGV-ES]), Brazil.


METHODS

Study design

A cross-sectional, school-based epidemiological survey was performed with a sample of students enrolled in high schools in the RMGV-ES in 2016 and 2017. The RMGV-ES is home to approximately 48% (1.6 million inhabitants) of the state population, including about 148,000 adolescents aged 15 to 19 years.15 It comprises seven municipalities, with Human Development Index (HDI) ranging from 0.686 to 0.845.16 According to the Espírito Santo State Department of Education, as of 2014, the RMGV-ES had 168 high schools and 65,763 regularly enrolled students. Secondary education concentrates students aged 15 to 19 years in Brazil (Law 9,394/1996), corresponding to the group covered in the present study. This study used data from the surveillance of risk factors for diseases and injuries in adolescents aged 15 to 19 years in the RMGV-ES database, which aimed to measure the exposure of this group to different risk behaviors, diseases, and injuries that can affect their full development and impact their physical and mental health.

Inclusion and exclusion criteria

All students aged 15 to 19 years, regularly enrolled in state public or private high schools located in the RMGV-ES, and bearing no cognitive, auditory, or visual impairment that could prevent their active participation in the study, were eligible to participate. All the students who agreed to participate, who signed a consent and/or assent form, and who effectively answered the research instrument were included.

Sampling and data collection

The sample was stratified according to municipality. Proportional sample quotas were calculated to match the distribution of students per municipality of the RMGV-ES. In order to determine the sample size, the following were considered: prevalence of bullying of 56.9% for victimization and 38.5% for aggression, based on Silva et al.,9 95% confidence interval (95%CI), margin of error of 2.5%, and design effect of 1.5.

According to the proportion of students enrolled per municipality of the RMGV-ES (Cariacica = 19.3%; Fundão = 0.5%; Guarapari = 5.8%; Serra = 23.2%; Viana = 3.7%; Vila Velha = 22.7%; Vitória = 24.8%), the number of adolescents to be interviewed was defined. All schools providing high school education in the RMGV-ES in 2014 were numbered consecutively in each municipality. A simple random sample selection of schools was performed using the BioEstat version 5.4 program according to the target proportion of each municipality.

The final sample included 2,293 participants. Data collection was carried out by trained investigators in 54 schools (43 public, 11 private). Students completed a closed interview on a laptop computer during selected classes, in the morning and afternoon shifts. Specific software for online and offline data collection was developed.

Instruments and variables

The factors analyzed in this investigation included sociodemographic variables - age (15 to 19 years), gender (male, female), race/skin color (Caucasian, black, brown, Asian, indigenous) - type of school (public, private), school shift (morning, afternoon), and high school grade (1st, 2nd, 3rd, 4th grades).

In order to assess bullying, a modified Brazilian version of the Olweus Bully/Victim Questionnaire4 was adopted, which includes the victim and bully versions. The modified version does not present a definition of bullying to the students being surveyed. Victims and bullies are categorized according to the self-reported frequency of attitudes that characterize these behavior in the past 12 months. The type of manifestation is also recorded, namely, physical, verbal, social, or psychological, sexual, and cyberbullying.4,8,17-19 The questionnaire is a Likert-type scale with six response options (0 = never happened; 1 = happened once or twice in the past 12 months; 2 = happened three to six times in the past 12 months; 3 = happened once a week; 4 = happened several times a week; and 5 = happened every day), for a total of 48 questions. Of these, 24 items investigate the frequency of victimization in the past 12 months, and 24 items relate to aggressions practiced in the same period.4,8,18

The psychometrics properties20 of the adapted instrument were analyzed. They revealed, for the items of the victimization scale, a mean ± standard deviation (SD) of 0.423±0.537, and, for the aggression scale, a mean ± SD of 0.261±0.432. The correlations between each item and the overall value of the scale ranged between 0.284 and 0.689 on the victimization scale and between 0.320 and 0.720 for aggression. Cronbach alpha values = 0.887 (victimization scale) and 0.888 (aggression scale) were obtained. The internal consistency of the total items was considered satisfactory for this sample (Table 1).




We proposed an alternative classification in order to capture an in-depth view of how students perceive the issue of aggression/victimization. Thus, the term maltreatment4 (maus tratos, in Portuguese) was used to refer to violent acts among students happening frequently, up to six times in the past 12 months; and the term bullying was used to refer to aggressive behaviors among students occurring more than six times during the past 12 months. The classification method was adapted from Fischer et al.4

The construct proposed in this study considered the frequency of the episodes, both in the victim and bully version. For each respondent, regardless of the number of repetitions of the response, the highest frequency defined the classification. The subjects were categorized as: never happened, maltreatment (once to six times in the past 12 months), frequent bullying (once a week), very frequent bullying (several times a week) and daily bullying (every day). Based on this classification, the total of victimization and perpetration was constructed. The types of bullying behavior (victim and bully) were also considered, and were categorized into physical, verbal, social or psychological, sexual, or cyberbullying. The concordance between victimization and aggression was also analyzed.

Pilot study

A pilot study was carried out to ensure uniformity and standardization of data collection. The form was tested using the test and retest method in two steps, with a 21-day interval between the administrations. In the first phase, it was administered to 46 adolescents aged 15 to 19 years (not included in the main study); and in the second phase, it was re-administered to the same students in order to verify inconsistencies regarding the adolescents' responses. When submitting the instrument questions to the kappa test,21 adjusted for prevalence, the results showed a variation from 0.71 to 1.0. The McNemar test was also used in order to assess the existence of any discordance tendency between the first and the second administration of the instruments. No statistically significant discordance was detected except for the variable "I was cursed," which demonstrated a tendency of significant discordance (p = 0.01).

Statistical analysis

Data were analyzed using SPSS version 21.0. Absolute and relative frequencies were calculated for all the variables analyzed. The Pearson's chi-square test was used to test the associations between bullying and sociodemographic characteristics. A multinomial logistic regression analysis was performed including the outcome variables victimization and aggression and the independent variables with statistical significance < 5% in the final model. The never happened category was used as reference, whereas adolescent age and high school grade worked as covariates.

Ethical aspects

The study was approved by the health research ethics committee at Universidade Federal do Espírito Santo (UFES) (Resolution 971,389/2015) and met all ethical parameters. All participants signed an informed consent or assent form. Participants younger than 18 years of age were also required to have the informed consent form signed by their parents or legal guardians.


RESULTS

The final sample included 2,293 adolescents, with 60% of females, 55.6% aged 15 to 16 years (mean age: 16.42±1.141), and 45.4% reporting brown race/skin color. Among the participants, 88% were public school students, 82.1% were enrolled in the morning shift, and almost half (47.7%) attended the first year of high school (Table 2).




The distribution of bullying victimization/perpetration and of sociodemographic characteristics among adolescents is shown in Table 2. Younger adolescents reported greater bullying victimization than their older counterparts (p = 0.009), with no age-specific distribution among bullies (p = 0.282). The same occurred in relation to school grade, which is linked to age. Female students reported greater involvement in bouts of maltreatment, while males reported more involvement in episodes of bullying. As far as shift is concerned, the presence of maltreatment and bullying was higher in the morning shift (p = 0.005), with no differences in distribution per shift among the victims. Private school students reported more victimization and aggression from maltreatment and bullying than their public school counterparts. It was observed that race/skin color did not impact the differential distribution of events.

Table 3 shows the final regression model. In the victimization model, the age variable was included as covariate; and in the aggression model, the variable high school grade was included as covariate, since these variables presented a significant correlation with victimization and aggression (R2 = 0.596; p < 0.001). Based on the category never happened, and considering victimization, the following were associated with bullying: school network (p = 0.030, odds ratio [OR] = 0.632, 95%CI 0.417-0.957) and gender (p < 0.001, OR = 0.624, 95%CI 0.485-0.802). The variable high school grade was excluded from the final model of victimization because of the interaction with the variable age. Regarding aggression, school network was significantly associated with maltreatment (p < 0.001, OR = 0.500, 95%CI 0.355-0.706) and bullying (p < 0.001; OR = 0.508; 95%CI 0.351-0.734), whereas gender (p < 0.001, OR = 0.441, 95%CI 0.353-0.734) and school shift (p = 0.039, OR = 1.350, 95%CI 1.015-1.795) were statistically associated with bullying only.




Table 4 shows the results for bullying victimization. When observing the total number of victims, 84.3% (n=1,926) of the adolescents reported some victimization in the past 12 months. Out of this total, 41% reported having been bullied.




When analyzing the types of bullying victimization, we observed that verbal victimization was the most significant, involving 33.8% of students, followed by social victimization (21.8%) and physical victimization (15.1%).Sexual bullying and cyberbullying were identified in 2 and 2.7% of the victims respectively.

In verbal victimization, the most frequent item for reporting maltreatment (31%) and bullying (24.5%) was "I was cursed." The item of social victimization "Somebody laughed and pointed at me" concentrated more reports of bullying (10.4%) and the item "Somebody tried to make others dislike me" had more reports of maltreatment (25.9%). As for physical victimization, it is noted that the most frequent response was destruction of property ("Somebody broke my stuff") both for maltreatment (27.5%) and for bullying (7.8%). Regarding sexual harassment, 1.9% (n=44) reported having undergone some of bullying. As for cyberbullying, 2.7% (n=64) chose responses that characterized bullying victimization.

Regarding aggressive behaviors, a total of 40.4% (n=920) reported aggression for maltreatment and 29.1% (n=662) for bullying over the past 12 months (Table 5). Regarding the type of bullying, the verbal type (23.1%) was most often perpetrated, followed by social (16.9%) and physical (8.7%) bullying. Sexual bullying and cyberbullying were detected in 0.9% (n=18) and 1.8% (n=40) respectively.




The item of verbal aggression "I cursed another classmate" was the most frequent (maltreatment = 24.3%, bullying = 19.2%). Social bullying was practiced by 8.7% through ignoring a classmate. As for physical bullying, most students reported having "punched, kicked and shoved someone." In the sexual type, it was observed that 0.8% (n=17) of the students practiced bullying according to the question "I sexually harassed someone." In cyberbullying, 4.6% (n=105) chose responses that characterized maltreatment and 1.8% (n=40) indicated having practiced cyberbullying.

Table 6 shows the results of concordance analyses regarding victimization and aggression behavior among adolescents. When analyzing the types of behavior, the results show that for the physical type (physical victimization vs. physical aggression) 54.5% of the adolescents presented concordant answers among the different categories of response. Discordant values of physical types confirm that 33.6% of the answers showed greater involvement in victimization by the students. Among these individuals, 82% reported having been victims to some degree, but without practicing any type of bullying against another student. Regarding participation as aggressor in a more intense way (more aggressor), 11.9% of students were involved more often as perpetrators than as victims. These patterns were similar to that identified in verbal and social types. Among the discordances, in each type analyzed it is also possible to note a significant number of responses that indicated the involvement of the students only as victims. Concerning sexual and cyberbullying, there was a high percentage of concordance among the respondents, 93.4 and 85.7% respectively, with the highest number of responses in the category never happened.




The comparison of total victimization vs. total bullying revealed that 49.6% had the same participation profile across categories in both behaviors. When analyzing discordance, it is noted that 37.5% of the students showed greater participation as victims. Of these, 50.9% revealed to have been only victims, without having practiced any type aggression against another student, whereas 31.1% of the students reported that they were just bullies.


DISCUSSION

This study explored bullying behaviors and associated factor in a population of students enrolled in public and private high schools. The results contribute locally and nationally with information on bullying victimization and aggression, allowing comparison with the findings of other Brazilian and international studies. In addition, the study investigated an unexplored topic in the Brazilian population, especially the concordance analyses between victimization and aggression.

Considering the results of the regression analysis, it was verified that the sociodemographic variables gender and school network remained associated with the adjusted final model of victimization, while the variables gender, school network, and shift remained associated with the final model of aggression. Regarding the gender variable, it was possible to observe that being a female adolescent prevented the occurrence of bullying in approximately 37 and 55%, respectively, for victimization and aggression. These findings corroborate the results or previous Brazilian investigations3,9,22,23 showing greater involvement in aggressive behaviors by male adolescents. This might be related to their higher inability to deal with conflicts, feelings of threat, or difficulties in discerning aggression jokes that can also express socially expected behaviors that reproduce the macho social model, especially in aggressive practices.1,4,6,22,24

As far as school network is concerned, the results of this study revealed that the public school system is related to a lower chance of aggressive behavior. The findings show that studying in the public network prevented in about 40% the experience of being bullied, and in 50% the perpetration of maltreatment and/or bullying. Although the number of students enrolled in the private school system of this sample was lower than the number of public school students, the results are in agreement with a study carried out in Argentina, which detected more bullying in private schools (28.3%) compared to public schools (17.2%).25 These finding contradict the common knowledge that tends to associate higher rates of school violence with socially and economically challenged communities.6

As for the findings associated with school shift, few studies have addressed the issue, especially in high school. Santos et al.23 investigated this relationship in Brazilian school children aged 13 to 17 years; however, the school shift did not show statistical significance. Therefore, further studies are needed to investigate the role of shift in the occurrence of bullying, given that we detected a 35% greater chance of aggression in the morning shift in relation to the afternoon shift. In the meantime, one possible explanation for this is the greater number of students in the morning shift.

Still considering sociodemographic characteristics, the age covariate used in the victimization model indicated a 15% reduction in bullying victims with each progression in school grade. Also, the high school grade covariate used in the bullying model (aggression model) demonstrated higher levels of bullying among younger students in earlier high school grades. The formation of social groups, typical of early adolescence, seems to be a possible explanation, since this grouping can marginalize some students or even rely on violence as a form of inclusion in the groups.4,26 Also, it may reflect the fact that as students progress in age and school grade, the frequency of aggressive behaviors tends to decrease.4,26

Based on the classification adopted - aggressive behavior among students -, a high percentage of bullying involvement was found in this group of adolescents, both as victims (41%) and as bullies (29.1%). This is worrisome and similar to other Brazilian studies,6,9,19 (even some using different instruments6,24), and reveals the severity of the phenomenon and the potential for harm. Conversely, the fact that other studies detected lower percentages22,27 may be reflecting the multi-causality of the phenomenon, variation in the instruments, evaluation strategies and methodologies employed, or even different family habits, and cultural, age, gender, and other peculiarities.28,29 Such differences may generate variations in the frequency and in most common types of bullying behavior in different countries, regions, and schools.9,28,30

The most prevalent type of bullying was verbal (victim = 33.8%, bully = 23.1%), followed by social (victim = 21.8%, bully = 16.9%) and physical (victim = 15.1%, bully = 8.7%) bullying. Similar results have been reported by previous studies, which indicated the verbal type as the most prevalent in adolescence.2,4,5,22,31 This type of bullying can go unnoticed by adults, who tend to seek solutions in situations of physical bullying. However, just like the physical type, verbal bullying is capable of causing serious harm to the health of those involved.2 Among the forms of manifestation of verbal bullying, the most frequent were name-calling (victim = 24.5%, bully = 19.2%), as also found by Fischer et al.4 Name calling using characteristics that represent the weaknesses of those involved may be used to avoid punishment.2,4 Like us, other researchers have found a greater frequency of the social type over the physical type.25 However, in general, verbal and physical9,27 types are predominant over the social type in scientific studies. This may be due to the greater visibility of the physical type2 or the non-manifestation of those involved, reducing the perception of occurrences by adults.1

The sexual type (victim = 2%, bully = 0.9%), although less reported, focused on sexual harassment. The result obtained is similar to the findings by Moura et al.22 The few reports about this type of bullying may be associated with the possibility of it being expressed through words and interpreted as verbal bullying rather than sexual,22 in addition to involving different power relations.32 In relation to cyberbullying (victim = 2.7%, bully = 1.8%), low percentages were observed. Nevertheless, this type of bullying spreads quickly, which can increase the victims' suffering.17

A similar percentage of maltreatment reports was observed (43.3% among the victims and 40.4% among the perpetrators). However, physical maltreatment was more frequent than social maltreatment, a situation that is contrary to that of bullying. This can be explained by the difficulty in assuming the frequency of physical involvement in the behavior, either because it is an act of violence or because of the embarrassment in reporting the physical episode. Also, students often interpret the involvement as play,4,13 which may not be perceived as aggression.

When the victimization and bullying behaviors were compared, it was possible to confirm expressive concordance in the simultaneous involvement as victim and bully, which may show absence of an absolute dividing line between these behaviors. This could also express a defense mechanism.22 However, in the present study, concordant values include individuals who stated they were not involved in any of the behaviors. In the comparisons between victimization and aggression behaviors, the number of students who claim to be only victims, without involvement as perpetrators, stands out. Such situation is recurring in all comparisons. This may be related to the victims' tendency of not reacting to the aggressions suffered and to the negative feelings generated by victimization.1,4 Another feasible explanation may be that the victims do not perceive themselves as bullies, and they can express their feelings of rejection through aggressive behavior without even realizing it.9

Different authors have concretely demonstrated the potential harm of bullying to the health of those involved, ranging from simple health problems to serious socialization difficulties, including the risk of psychological and physical damage (self-mutilation and suicide). These consequences can be carried on to adulthood in several manners.33,34 The occurrence of bullying in schools is a worldwide phenomenon that affects public and private schools. School communities should urgently adopt effective prevention measures in order to promote a culture of non-violence, making the school environment a place of protection and integration.1,13,23

Some limitations still have to be considered. Data were collected through self-report at a single time point, which can produce socially expected responses and differences in interpretation. Age groups can influence judgment and ways of self-perception, repeating patterns generated by the trivialization of violence. Despite the limitations related to the instrument itself, concerning the proposed measurement period, this instrument was quite effective according to the pilot study.

In summary, the adolescents identified as victims did not generally attack other students, i.e., did not identify themselves as perpetrators. The high prevalence of maltreatment and bullying among adolescents, as well as the different types of manifestation and factors associated with aggressive behaviors in the present study, are worrisome, giving visibility to the phenomenon outside the American and European contexts. The study brings to light the risks students are exposed to within school settings, a primarily protective space that should not replicate social violence. For that, systematic interventions are necessary, to promote health and prevent violence.


ACKNOWLEDGEMENTS

We are grateful to Escola Nacional de Saúde Pública Sérgio Arouca, Fundação Oswaldo Cruz (ENSP-FIOCRUZ), Instituto Federal do Espírito Santo (IFES), Campus Vitória, and Fundação de Amparo à Pesquisa do Espírito Santo (FAPES), for their support in carrying out this research.


DISCLOSURE

The authors report no conflicts of interest.


REFERENCES

1. Lopes-Neto AA.[Bullying--aggressive behavior among students].J Pediatr (Rio J).2005;81:S164-72.

2. Bandeira CM, Hutz CS.Bullying: prevalência, implicações e diferenças entre os gêneros.Psicol Esc Educ.2012;16:35-44.

3. Teixeira VA,Coladith EV,Jacomel RL,Ulbricht L,Neves EB.Bullying nas escolas municipais de Curitiba-PR: um problema de saúde pública.Rev Uniandrade.2013;14:25-43.

4. Fundação Instituto de Administração (FIA).Bullying escolar no Brasil - Relatório final [Internet].2010 Mar[cited 2019 Jan 30].http:/www.ucb.br/sites/100/127/documentos/biblioteca1.pdf

5. Brito CC,Oliveira MT.Bullying and self-esteem in adolescents from public schools.J Pediatr (Rio J).2013;89:601-7.

6. Oliveira WA,Silva MA,Silva JL,Mello FC,Prado RR,Malta DC.Associations between the practice of bullying and individual and contextual variables from the aggressors' perspective.J Pediatr (Rio J).2016;92:32-9.

7. Silva ABB.Bullying: mentes perigosas nas escolas.São Paulo:Globo Livros;2015.

8. Petroceli AW.Associação entre bullying e predisposição de respostas a estímulos em adolescentes [monograph].Porto Alegre:Universidade Federal do Rio Grande do Sul (UFRGS);2013.

9. Silva CE,de Oliveira RV,Bandeira DR,de Souza DO.Violência entre pares: um estudo de caso numa escola pública de Esteio/RS.Psicol Esc Educ.2012;16:83-93.

10. Mattos MZ,Jaeger AA.Bullying e as relações de gênero presentes na escola.Movimento.2015;21:349-61.

11. World Health Organization (WHO).La salud de los jóvenes: un reto y una esperanza.Genebra:WHO;1995.

12. Brasil, Ministério da Saúde, Secretaria de Atenção à Saúde, Departamento de Ações Programáticas Estratégicas.Saúde do adolescente: competências e habilidades [Internet].2008[cited 2019 Jan 30].http:/bvsms.saude.gov.br/bvs/publicacoes/saude_adolescente_competencias_habilidades.pdf

13. Malta DC,Silva MA,Mello FC,Monteiro RA,Sardinha LM,Crespo C, et al.[Bullying in Brazilian schools: results from the National School-based Health Survey (PeNSE), 2009].Cien Saude Col.2010;15:3065-76.

14. Pigozi PL,Machado AL.Bullying during adolescence in Brazil: an overview.Cien Saude Colet.2015;20:3509-22.

15. Instituto Brasileiro de Geografia e Estatística (IBGE).Censo demográfico 2010 [Internet].[cited 2019 Jan 30]. http://censo2010.ibge.gov.br?

16. Atlas do desenvolvimento Humano no Brasil.Ranking - Espírito Santo [Internet].2010[cited 2019 Jan 30]. http:/www.atlasbrasil.org.br/2013/pt/ranking

17. Silva ABB. Bullying: cartilha 2010 - Projeto Justiça nas Escolas [Internet].2010[cited 2019 Jan 30]. http:/www.cnj.jus.br/images/programas/justica-escolas/cartilha_bullying.pdf

18. Silva NT.Associação entre comportamentos de oposição em adolescentes e arousal: avaliação por meio de resposta cutânea simpática [dissertation].Porto Alegre:Universidade Federal do Rio Grande do Sul (UFRGS);2014.

19. Zottis GA,Salum GA,Isolan LR,Manfro GG,Heldt E.Associations between child disciplinary practices and bullying behavior in adolescents.J Pediatr (Rio J).2014;90:408-14.

20. Vizini ST.Evidence of the validation of the Olweus Bullying Questionnaire victim and aggressor version for brazilian adolescents [dissertation].Porto Alegre:Universidade Federal do Rio Grande do Sul (UFRGS);2017.

21. Landis JR,Koch GG.The measurement of observer agreement for categorical data.Biometrics.1977;33:159-75.

22. Moura DR,Cruz AC,Quevedo Lde A.Prevalence and characteristics of school age bullying victims.J Pediatr (Rio J).2011;87:19-23.

23. Santos JA,Cabral-Xavier AF,Paiva SM,Leite-Cavalcanti A.[The prevalence and types of bullying in 13 to 17 year-old Brazilian schoolchildren].Rev Salud Publica (Bogota).2014;16:173-83.

24. Guimaraes NM,Pasian SR.Aggressiveness in adolescence: experience and expression of anger.Psicol Estud.2006;11:89-97.

25. D'Angelo LA,Fernández DR.Clima, conflictos y violencia en la escuela.Buenos Aires:Unicef, Flacso;2011.

26. Seixas SRP. MMComportamentos de bullying entre pares - bem estar e ajustamento escolar [thesis].Coimbra:University of Coimbra;2006.

27. Rech RR,Halpern R,Tedesco A,Santos DF.Prevalence and characteristics of victims and perpetrators of bullying.J Pediatr (Rio J).2013;89:164-70.

28. Berger KS.Update on bullying at school: science forgoten?Dev Rev.2007;27:90-126.

29. Alckmin-Carvalho F,Izbicki S,Fernandes LFB,Melo M.Strategies and instruments for the identification of bullying in national studies.Aval Psicol.2014;13:343-50.

30. Liang H,Flisher AJ,Lombard CJ.Bullying, violence, and risk behavior in South African school students.Child Abuse Negl.2007;31:161-71.

31. Silva F,Dascanio D,Valle TGM.O fenômeno bullying: diferenças entre meninos e meninas.Reflexão Ação.2016;24:26.

32. Ferreira WB.Vulnerabilidade à violência sexual no contexto da escola inclusiva: reflexão sobre a invisibilidade da pessoa como deficiência.Rev Iberoamericana Calidad.2008;6:120-36.

33. Carney JV,Jacob CJ,Hazler RJ.Exposure to school bullying and the social capital of sixth-grade students.J Humanist Couns.2011;50:238-53.

34. Kowalski RM,Limber SP.Psychological, physical, and academic correlates of cyberbullying and traditional bullying.J Adolesc Health.2013;53:S13-20.