This is part 2 of my 4-part series about studying gender bias in science (See part 1, part 3).
For studies using existing data, we look at information that is already available, and learn from the information through data analysis. The difficulty in these studies is that because you are not in control of how the information came about, you cannot control all the different factors that can contribute to your results. Data analysis becomes very important.
The two papers I am reviewing in this post are about the percentage of female presenters at academic conferences. Why study gender bias in a conference setting? Scientific conferences are where scientists meet and share their latest research findings. The lack of women presenting at a conference results in low visibility of established female researchers. This could give younger female scientists the impression of the lack of women in the field and discourage them from staying in the field. In addition, gender bias could mean that we are missing out on outstanding research done by female scientists.
First, let’s look at Stag Parties Linger: Continued Gender Bias in a Female-Rich Scientific Discipline by Isbell, Young, and Harcourt, published by PLOS ONE in November 2012. In the study, data from 21 annual meetings of the American Association of Physical Anthropologists (AAPA) were reviewed and analyzed. The authors focused on the percentage of female scientists doing poster, oral (better than poster), and symposium (invited, better than oral) presentations on Primatology at the AAPA conferences. Immediately I had some difficulty with this paper, because the authors did not state up front exactly what their research questions and hypotheses are. It is hence hard to tell what they are analyzing, what they should compare the data to, and what I should consider important in the paper. I also have problems with their data selection criteria among a few other things. I will include that as supplementary reading at the end of this post for those who are interested.
The authors found that women gave significantly more poster than oral presentations, while for men this is not the case: Women – 599 posters vs. 313 talks; Men – 283 talks vs. 253 posters. Why? The authors suggested:
One possibility for the bias in favor of women giving posters is that it happens not at the self-selection stage but by others during the development of meeting programs. However, data on presenter requests and decisions from the program committee for the last two years show that men were 12% more likely than women to request talks over posters, although this difference is not statistically significant with only two years of data… In addition, men’s and women’s requests to present talks were denied at a statistically similar rate… Nonetheless, non-significant trends in assigning posters to those who preferred to give talks (22% for men vs 29% for women) suggest at least the possibility of some additional bias against women occurring at the selection stage.
I am not sure how the authors managed to write this using 2 years of data especially when no significance was shown. If you have lots of data, perhaps it is possible to argue the trend – but data from 2 years cannot really tell us anything.
The most important figure and data analysis in this paper is Figure 2. In this figure, the authors tried to demonstrate that when the symposia were organized by committees with only female organizers, or by those with organizers from both genders, the percentages of symposium presentations with women as first authors (because typically the first author is the most significant contributor, and most likely the presenter, but this varies from field to field) are not significantly different from the overall percentage of poster and oral presentations with women as first authors – represented by the solid black line. What is striking is that when the committees had only male organizers, the proportion of presentations with women as first authors dropped significantly to 28.8%.
Figure 2. Proportion of women as first authors of posters, talks, and symposia at AAPA meetings. The average proportion for all presentations with women as first authors over a 21-year period of annual meetings of the American Association of Physical Anthropologists is indicated by the solid black line. F-Org. Symp.: symposia organized by women only; F/M Org. Symp.: symposia organized jointly by women and men; M-Org. Symp.: symposia organized by men only.
There are several problems I can find with this analysis. First of all, I disagree with how the authors thought that career stage effects are irrelevant here. The black line groups female presenters at all stages together – from perhaps undergrads, PhDs, all the way to established researchers. Typically, invited speakers are established researchers, so if there is a small pool of established female researchers to begin with, then the issue is with the lack of established female researchers (in itself, also a problem), not with the selection process (bias/discrimination). Secondly, the authors compared ratios based on the average from all 21 annual meetings. This is very risky, as a small number of years with large disparities could easily affect the overall average (granted, the gap does seem large). An average also does not reflect changes throughout the years (are things getting better/worse?). The authors mentioned they did compare the averages over 7 year increments – why 7 years? Would the results change if we choose another increment? This decision is very arbitrary.
In the end, the authors concluded that there is a gender bias against women in the field of Primatology. I personally don’t think that the analysis in this paper was done well. This is not to say that the authors were wrong, but bad analysis means that it is not possible to tell if the results indeed reflect reality, and bad analysis does not allow us to pin point the problem and discuss possible causes.
The next paper, Fewer invited talks by women in evolutionary biology symposia by Schroeder et al, published in the Journal of Evolutionary Biology just last month, was done much better! The authors analyzed data from the European Society for Evolutionary Biology (ESEB) Congress in 2011 (a recent meeting where all abstract submissions were accepted), and also from 2001-2011, looking at the sex ratio of presenters (poster, regular, invited, or plenary). The authors right away talked about their hypotheses for the project:
We hypothesize that because the scientific achievements of women may be less visible than the achievements of men, female scientists may be overlooked more often for invitations to talk. If this is true, we expect the sex ratio of invited speakers to be biased towards males, even after accounting for career stage and the population sex ratio of the research field. The sex ratio of speakers at a symposium can also depend on the genders of the symposium organizers, with fewer women speaking in male-only organized symposia. We therefore expect that symposia organized only by men will have fewer female invited speakers than symposia that have at least one female organizer.
The authors started by establishing the two baselines for the purpose of comparison, hence controlling for career stage effect and presenter research quality. 1) the faculty sex ratios from the Evolutionary Biology departments at the world’s top-10 universities for life sciences and 2) the sex ratios of first and last authors of research articles published in Nature and Science. Last but not the least, they also included the sex ratios of faculty in biosciences in the UK, and sex ratio of faculty in science and engineering across the EU, for comparison.
Looking at ESEB 2011 specifically, the authors found that the sex ratio of the plenary speakers (also invited, but plenary speakers are usually prestigious) did not differ significantly from the sex ratio of all other oral presenters, or from that of all regular speakers (presenters?). But, the sex ratio of invited speakers was biased toward men when compared to all other presenters and when compared to all regular speakers (the terms got a bit confusing here). The most striking discovery was this:
Although 23% of all initially invited speakers (including those that declined) were women, only 15% of the realized invited speakers were women. This reduction was because 50% of invited women declined talks compared to 26% of invited men…
This group also looked at whether there is any correlation between females in the symposia organizing committee and the percentage of female invited speakers:
There was no association between the presence or absence of female organizers and the respective sex ratio of their invited speakers, contrasting with the findings of Isbell et al.
Then, they started to look into the data from past ESEB congresses (2001-2011).
Randomizations showed that the sex ratio of realized invited speakers (15% women) was lower than baseline populations of early–mid career stage scientists (including first authors in top-tier journals), but similar to senior scientists (Professor and last authors in top-tier journals; (Fig. 4)). However, the 23% of initially invited speakers who were women (and of whom a larger proportion of women than men declined to speak) was lower than baseline populations of early–mid career stage scientists (Lecturers & Fellows) but did not differ from Professors or authors in top-tier journals (Fig. 4).
So some good news and bad news from this paper. Good news – the sex ratio of invited speakers at ESEB 2011 did not differ from that of Professors or authors in top-tier journals (so this points to the issue of a small pool of female researchers to invite from). Also, whether or not there are females on the symposia committee does not affect the sex ratio of invited speakers at ESEB.
Bad news – 50% of the invited female speakers declined invitations, leading to the very low number of final, realized female invited presenters. Why? Could it be how females tend to not like self-promotion? Have self-assessment bias? Maybe the problem lies with Child care (which was not available at ESEB 2011)? Lack of travel funding? Why don’t plenary speakers have the same problem? The authors provided some references in their article that are worth reading.
It was enjoyable reading these two articles side-by-side, giving me a chance to compare how the two studies were done. I think the lesson for me here is that the study of gender bias using existing data can get really complicated. It is important to make attempts to control other factors that might affect the results, instead of giving yourself arbitrary reasons not to do so. Also, studies like these are done for a specific field, and we must recognize that gender bias might exist in other fields that were not studied, such is the limitation of these papers.
Next time in this series, I will be reviewing: Science faculty’s subtle gender biases favor male students by Moss-Racusina et al, and The Matilda Effect in Science Communication: An Experiment on Gender Bias in Publication Quality Perceptions and Collaboration Interest by Knobloch-Westerwick, Glynn, and Huge. It will likely take me 1-2 weeks to write that up, so stay tuned!
Isbell L.A., Young T.P., Harcourt A.H. & Lambert J.E. (2012). Stag Parties Linger: Continued Gender Bias in a Female-Rich Scientific Discipline, PLoS ONE, 7 (11) e49682. DOI: 10.1371/journal.pone.0049682.g002
Schroeder J., Dugdale H.L., Radersma R., Hinsch M., Buehler D.M., Saul J., Porter L., Liker A., De Cauwer I. & Johnson P.J. & (2013). Fewer invited talks by women in evolutionary biology symposia, Journal of Evolutionary Biology, n/a-n/a. DOI: 10.1111/jeb.12198
Postscript 1: Here are some other reviews of the Isbell, Young, and Harcourt paper
I would also suggest that you read Athene Donald’s blog post All Male Invited Speakers? It’s Complicated! on the Schroeder et al results. The most interesting part is the discussions in the comment section (Athene tweeted about this and invited people to comment on the post/paper).
Julia Schroeder and Deborah Buehler wrote a post about their results, Women’s contribution to science goes unheard.
Postscript 2: There are some additional anecdotal discussions on blogs regarding gender bias at other conferences. Unfortunately(?), in some cases, the discussion of the lack of women at conferences became a discussion of statistical methods. This further speaks to the difficulty and importance of properly analyze existing data.
Interestingly, not all AAPA presentations are included. They authors listed, in their methods section, their selection criteria:
- Include titles in sessions devoted to to primate behavours/ecology
- Exclude titles in sessions devoted to primate skeletal biology/evolution
- Exclude titles in which non-human primates were used only as models to address hominin behaviour or evolution
- But hominin hahaviour/evolution was included when looking at symposia
2 and 3 are because these areas do not have strong female representation (What does that mean? Is that a good explanation?). 4 is because “they are interested in the behaviour of the organizers than the relevance of the topics” (but wouldn’t that be interesting for posters and oral presentations?). The rationale is not clear to me, so at this point it feels a bit cherry-picking.
While the authors’ main focus was the American Association of Physical Anthropologists (AAPA) annual meeting, the authors mentioned results from other meetings (American Society of Primatologists, American Society of Mammalogists), which was really distracting for me. I wasn’t sure how data selection was done for the other meetings, and if the other meetings are also attended by the same pool of people.