Comparative Methodological Critique

The aim of this paper is to compare two academic research papers, one informed by qualitative and another one by quantitative research designs with focus on the methodological factors. Both papers describe working lives and attitudes of gay and lesbian workers in the UK and USA, possible consequences of disclosure of sexual orientation on their working relations and organisational arrangements towards equality and diversity of working practice. These research projects add to the growing number of studies which shed light on the sensitive nature of homosexuality in the workplace and anti-discrimination policies and practices that organisations deploy to create a more inclusive working environment. Governed by the difficulty of access to gay and lesbian population both in UK and USA, and its â€Å"hidden† nature both teams chose their own strategy in unveiling the nature of work attitudes of gay and lesbian employees and demonstrating the progress organisations have made or are making towards the establishment of good practice. st paper (qualitative) The paper deals with identifying good organisational practice concerning equality, diversity and sexual orientation in the workplace, and considers any changes following the introduction of Employment equality (Sexual Orientation) regulations 2003. It also aims at addressing the gap between equality policy and practice which provides the foundation for further analysis of the significant shortfall in knowledge relating to the experience of LGB (lesbian, gay and bisexual) employees in UK organisations. Even today, the issue of sexuality still remains a sensitive one, despite the coming into force of legislation offering protection to LGB people. It is also the cause of unease for some employers in the modern world in discussing (not to mention dealing with! ) this issue, and consequential bullying, suffering and fear which make life and work unbearable for so many LGB people. So far progress has been made based on social justice and business case studies which means that this still â€Å"remains †¦an under researched area in which there have been very few case studies of particular organisations† (Colgan et al. 2007:591). Having adopted a phenomenological position in their design, the research team chose to carry out a longitudinal (2 year) qualitative study in 16 â€Å"good practice† case study organisations in the area of employment of LGB workers, to discover â€Å"the details of the situation†¦[in order to] to understand the reality† (Remenyi et a l. , 1998:35) of working relations in organisations with LGB workers. Also focusing on the ways that people make sense of the world especially through the sharing of experiences with others has dictated the use of a social constructionist framework (Bryman and Bell, 2007). The case studies involved the analysis of companies’ documentation and reports, trade union publications and websites to get an inside view of companies’ attitudes and practices. The main challenge of the project in data collection was â€Å"limited resources †¦ inability to specify a sampling frame† (Saunders et al. , 2003:170) which dictated the use of non-probability sampling. Therefore, snowballing sampling was applied as the most appropriate for this kind of research. The research team interviewed in depth 154 LGB employees who, prior to that, completed a short survey questionnaire. All completed the survey giving a complete census. However, due to â€Å"a lack of robust statistical evidence concerning the proportion of the UK population who identify as LGB† (Colgan et al. , 2007:591) and the difficulty of access to such people, it’s hard to say how representative this sample of LGB people is. Determined by these practical constraints, the research team adopted a mixed methods research strategy (case study/interviews) with various data collection methods that allowed them to have a cross-check against each other's results. Difficulty of accessing LGB employees also dictated the choice of analysis focusing only on â€Å"good practice† organisations. Hence, using survey, in-depth interview and secondary data enabled the team to combine the specificity of quantitative data with the ability to interpret perceptions provided by qualitative analysis. Though, there is evidence and justification by the research team of how respondents were selected, the project itself demonstrated a disproportioned balance of respondents not only in terms of gender (61,7% -men, 38,3% – women) but also in terms of ethnic division. This means that the balance tipped so far in terms of the depth of their research. The research team adapted an inductive research approach to accommodate the existing theories and findings as well as their own empirical studies, for example â€Å"that those in lower level and perhaps more difficult working environments are less likely to be â€Å"out†Ã¢â‚¬ ¦at work† (Colgan et al. , 2006a cited in Colgan, 2007:593). The paper also gives a good historical overview of how public and private sector organisations have made progress towards the â€Å"inclusion of sexual orientation within its organisational policy and practice† (Colgan et al. , 2006a cited in Colgan, 2007:593). This provides readers with a comparative overview of how the situation has changed following the introduction of the legislation. The research team identified the number of steps organisations took to follow â€Å"good practice† in relation to equality and diversity, at the same time outlining the differences in progress between public and private sector companies. In doing so, the authors also assert that the implementation gap between policy and practice still exists and requires further measures such as â€Å"the need to see policy championed, resourced and implemented by senior line managers† (Colgan et al. 2007). Whereas the Regulations empowered LGB people to step in and â€Å"challenge discrimination and harassment† (Colgan et al. , 2007:604), the key factor, as identified by the authors, â€Å"that prevented some respondents coming out at work †¦ the fear† (Colgan et al. , 2007) remained the main obstacle to LGB workers to blowing the whistle and reporting an incident. These findings are supported and justified throughout the paper by respondents’ statements and results of previous studies. The research team also defined the relation between the impact of the employment equality (SO) regulations, considered as independent variable, and such dependent variables like job satisfaction, being out at work, experience of harassment and discrimination, this in fact gives an idea of the existing problem in organisational practices. In terms of structure, language and appropriateness of referencing to other material this paper proves highly informative. It suggests that further research needs to be done in order â€Å"to gain a more â€Å"representative† picture of working lives of LGB employees† (Colgan et al. , 2007) as it only explored the â€Å"tip of the iceberg†, because the research project was able to examine only a fraction of the whole, and, unless an organisation adopts more effective and proactive leadership on equality and diversity, the legislation on its own will take only a small step towards the inclusivity of LGB people and the creation of harmonious equal working practices in UK organisations. The research project involved cross-sectional design that allowed the team to compare and contrast their findings derived from each of the cases. This is considered to be a great advantage of the project as it allowed the team to consider â€Å"what is unique and what is common across cases† (Bryman and Bell, 2007:64), thus facilitating an attempt to generalise their findings. The research team does not explicitly express their own opinions, allowing personal values to sway the conduct of the research and the findings deriving from it, this ensures that they acted in good faith and followed good practice. However, having conducted a survey following the implementation of (SO) Regulations 2003, this study failed to provide a sufficiently broad spectrum of opinion of how organisational culture has changed as a consequence of the legislation. Moreover, the research team did not specify whether any of the key informants or interviewees had worked in â€Å"good practice† organisations before the regulations came into force which would allow a comparative analysis of organisational practice to take place and, thereby, for data to be more representative. Much attention was focused upon good practice organisations in their study; however, there was a failure to demonstrate what was meant by â€Å"good practice†. This would have enabled readers to have a more explicit understanding of what expectations are from any organisation with LGB employees. In terms of access to respondents, although the team acknowledged the difficulty, they were not as resourceful as they might have been. Lack of robust information undermines the conclusions, arguably, brings into question the credibility of the findings as the research net was not cast widely or deeply enough. For the readership of this paper there is no ultimate enlightenment as the paper does not show any argument or advocacy that invite readers to the world of unexpected discovery; instead, it is rather a presentation or portrait. It also remains difficult to conduct a true replication of this study, even though majority of the organisations â€Å"were willing to be identified by name† (Colgan et al. 2007). 2nd paper (quantitative) This paper describes the relationship between reported disclosure of sexual orientation, anti-discrimination policies and top management support. It is also aimed at identifying work-related attitudes of gay and lesbian workers if such disclosure takes place and its effects on individual performance of gay and lesbian workers. At the time of the research very few empirical studie s had been conducted to investigate work attitude and disclosure of sexual orientation. Since the recognition of the gay and lesbian population within the workforce, inclusiveness of gay and lesbian employees in organisational diversity management policies was desperately needed. It has been argued that â€Å"an approach of workplace tolerance is needed† (Day and Shoenrade, 2000:347) which can contribute to the knowledge of human resource function when taking appropriate actions if conflict arises and to help create a more conducive environment for the disclosure of sexual orientation. Emphasising the importance of collecting facts and studying â€Å"the relationship of one set of facts to another† (Anderson, 2009:45) underpins their positivist paradigm. Focused on key unresolved questions such as â€Å"closeted homosexual workers will have a less positive work-related attitude† (Day and Shoenrade, 2000:346) the research team takes a deductive research approach to test three formulated hypotheses which together with research paper aims do not appear until well into the text. This can lead to uncertainty on the part of the readership. In terms of selection of respondents and its rationale, the research team fails to demonstrate the proportion of respondents relating to ethnics; also gender division appeared to be highly disproportioned with 485 gays and only 259 lesbians. Their chosen data collection methods (sampling, focus groups and a questionnaire) highlighted some problematic issues such as â€Å"identifying a representative sample of working lesbians and gays† (Day and Shoenrade, 2000:350) due to the sensitive nature of research and lack of current data on the lesbian and gay population. Great attention was paid to demonstrating their strategy in obtaining possibility sample. Having looked at various options, the sampling choice was justified on the basis of geographical location (USA Midwest) which it was thought would benefit in the existing research as it involves lesbian and gay population not surveyed before. A large sample would allow them to easily obtain a significant test statistic (Esterby-Smith et al. 2008), the research team sampled several resources, principally Human Rights Project (HRP), and then broadened them getting a reasonable response of 29% which â€Å"showed no statistically significant difference from the larger sample on the major variables† (Day and Shoenrade, 2000:351). However, such considerations cannot be viewed as justification for not following the principles of probability sampling and therefore team’s findings do not represent the whole gay and lesbian population of USA and cannot be generalised. In testing three formulated hypotheses the research team established three independent variables (disclosure of sexual orientation, presence of anti-discrimination policy and top management support) and 5 dependent ones (affective commitment, continuance commitment, job satisfaction, job stress and conflict between home and work) and selected a multivariate analysis to find a way of summarising the relationship between these variables and at the same time capturing the essence of the said relationship (Esterby-Smith et al. , 2008). Adopting objectivist ontology the research team aimed at analysing the relationships between these variables thus creating static view of social life (Bryman and Bell, 2007). However, there was a failure to specify which statistical test was used to demonstrate and examine interdependence between them. This, in fact, point to a lack of transparency of their project. Based on several example questions the team demonstrated it can be deduced that they were dealing with categorical data (ordinal and cardinal) and therefore the Spearman correlation test and regression analysis (linear regression model) were used to represent non-parametric data. This allowed them to measure the strength and the direction of association between the variables, and confirm whether there is any difference in the population from which the sample was drawn. The research team also used a qualitative research method. They conducted a focus-group discussion to ensure that â€Å"the construct of disclosure of sexual orientation be precisely defined and relevant to†¦ [this] population (Day and Shoenrade, 2000:351). This helped to ensure that the concept was precisely defined and relevant to the population. The lack of secondary data sources also explained the rationale of their chosen method of data analysis. However, had this data been obtained, it would have allowed to create a more comparative analysis. For each independent variable the research team demonstrated several example questions and interpretation of the results which provided readers with an overview of a critical analysis of the conclusions the research team came to. However, those who are not familiar with statistical data would find it hard to follow the argument as there was a failure to provide basic explanation of roman letters and include notes when referencing to the tables. Therefore, it is impossible to cross-check their results and to confirm their findings. Focusing on facts, and looking for causalities throughout their analysis, underpinned their positivist paradigm and reflected their research approach and strategy. Like any other quantitative researchers the team was trying to describe why things are, rather than how they are. They admit that the issue of causality should be discussed to achieve a better understanding of the relations of the variables. Nevertheless, the research team found the evidence to support portions of the three formulated hypotheses. They critically evaluated each one by giving their own suggestions and recommendations to organisations and HR practitioners on how to improve their working practice and establish the inclusiveness of gay and lesbian workers. The presence of modal verbs in the quantitative research once again emphasised the invariably implicit nature of this research often criticised by the qualitative researchers for its high level of assumptions (Bryman and Bell, 2007). Their data analysis and interpretation of their findings are well structured and presented. The ability of the research team to point out the pitfalls of their own project and critically approach their findings is considered to be a great advantage of this research paper. Conclusion The papers provide a solid foundation for further analysis in the field of equality and diversity in organisations with gay and lesbian workers. Governed by own choice for research strategy and approach to answer research questions, they were both affected by the same practical constraint during the research, extremely sensitive nature of sexual orientation and the â€Å"hidden† nature of the gay and lesbian population which led to that fact that probability sampling was impossible as there was no accessible sampling framework for the population from which the sample could be taken. Hence, one cannot confirm that both research projects were successful in generalising their findings beyond their chosen sample. Both research projects lack of robust evidence of the proportion of the UK gay and lesbian population and base their research projects on out of date information. This, in fact, stress the importance of secondary analysis to take place which would allow them to obtain good-quality and up-to-date data, and, as outlined by the quantitative research team, offer the opportunity for research to â€Å"compare an individual’ responses longitudinally† (Day and Shoenrade, 2000:361). It would also provide the opportunity to gain a more descriptive picture of gay and lesbian working lives and facilitate the emergence of a new data interpretation and theoretical ideas as new methods of quantitative data analysis are constantly appearing in the business research field. Unlike the qualitative research team which totally avoids any advice, the quantitative research team succeeded in providing various recommendations for organisations that can help them create a more inclusive environment, â€Å"this process could begin through education top management in the importance of the issue and the consequences of ignoring it† (Day and Shoenrade, 2000:360). In terms of contribution to the field unlike quantitative research the qualitative research project rather confirmed the existing findings and results of previous studies then introduced new information that would benefit and add to the existing knowledge. However, both research projects draw the parallel between UK and USA in terms of work attitude and practice in organisations with gay and lesbian workers pointing at lack of effective leadership and commitment of top management needed to create inclusive working relations in a contemporary business world. Both research teams admitted that further analysis will be required, perhaps, a combination of two research methods, as stated by the quantitative research team, â€Å"to gain a more â€Å"representative† picture of working lives† (Colgan et al. , 2007:606) of gay and lesbian workers.