Conducting Policy Audits: Opportunities and Challenges

This section summarises the opportunities and challenges identified in the consultation processes for the five policy audits commissioned by the Sexuality, Poverty and Law Programme in 2013. The audits were conducted by partner organisations in Brazil, China, India, South Africa and the Philippines and each covered a different sexuality-related theme. Some of the audits focus on heteronormativity as a guiding concept in the audit process.

For more information on the audits, download our Sexuality and Poverty Synthesis Report.

What is a policy audit?

A policy ‘audit’ is a systematic review of a set of policies and policy processes and usually focuses on a particular theme or area of policy. Conducting a policy audit can help you to understand more about the content and scope of existing policies, how they got there, how they are being implemented and who the key players are. This is likely to include the following activities:

  • A systematic review of existing policies in the area of policy that you are interested in
  • An analysis of policy-making processes
  • An analysis of how policies are being enforced

Why conduct a heteronormativity policy audit?

Heteronormativity is a term used to describe the assumption that men and women have fixed gender identities and that heterosexuality is the normal and natural union for sexual relationships and basis for the family. Undertaking policy audits using a heteronormativity ‘lens’ can give an insight into the extent to which assumptions about heterosexuality and the roles of men and women shape policy and the policy making process. This can be useful for understanding why certain people or groups are excluded from the benefits of social protection and poverty reduction strategies and whose rights are included and excluded in the national policy framework.

What should a policy audit look like?

There is no one format for a policy audit. This will depend on the audience and the purpose of the audit. For the heteronormativity policy audits, the following basic framework was agreed upon:

Section 1: Describe the context and the ways in which sexuality and gender norms are presented in the policy.

Section 2: Lay out the methodology you used and why you chose that approach, including any major problems you encountered and changes you made.

Section 3: Reflect on the actors, discourses, power and spaces, interrogating them through the questions above (LINK: Understanding policy processes)

Section 4: Illustrate your findings with case studies to bring the issues to life and to ensure that people affected by the policy are centrally involved in your analysis.

Section 5: Recommendations

How to conduct a policy audit: Methods

Policy audits can be carried out using a range of methods. The methods chosen will depend on the area of policy to be investigated, the data available and the expertise and experience in the research team. Listed below are examples of the methods used in the five heteronormativity policy audits, and the opportunities and challenges that were raised by the teams involved.

MethodDetailsOpportunitiesChallenges
Discourse AnalysisAnalysing written and verbal text to identify relations between discourses, power and context.Brings a wide range of material, such as policy documents, meeting minutes, speeches and newspaper articles into consideration.Can be very time-consuming. Requires familiarity with legal and policy language.
Content AnalysisStudying the content of written communication: ‘Who says what, to whom, why, to what extent and with what effect?’Can see how topics are talked about, different opinions on a topic, how debates take place etc without actually having to be at the events.Can be very time consuming, especially where documents are not in electronic form. Policy documents may not contain the language you are looking for. Key word searches may miss key information. Can be too simplistic and has a high margin for error.
Participatory Action Research (PAR)Seeks to understanding a particular issue/problem by changing it collaboratively and reflecting on that process of change.Participatory methods can allow individuals conducting the audit to reflect upon their own beliefs and biases around sexuality. Very useful for activists and organisations as it can be built in to their existing work.Time consuming. Can be difficult to gather people together and to keep them engaged through the different stages of the research. May be difficult to engage policy-makers in the process. Relies on knowledge and experience of the project and the process to be beneficial.
InterviewsConversations in which an interviewer attempts to elicit information and personal opinion from another.Can understand people’s lived experiences and gain insights into how policies affect peoples’ everyday lives. Can highlight differences between the assumptions of policy-makers and lived realities of those affected.Families may be wary of ‘outsiders’, unfamiliar with the research context, anxious about sharing their opinions, not able to speak freely in front of others. May be difficult to get privacy for discussion.
Participant ObservationThe researcher takes on the role of observer in a situation in which they are a participant, for example, in meetings or consultation processes.Can see how policy-making processes or enforcement take place. Possible to witness dynamics and relationships between individuals, groups, organisations and institutions. Can use your ‘insider’ knowledge to interpret situations.Access can be difficult: policy-making processes may be closed, badly advertised and certain groups may not be invited to attend. You may not have the right networks to be able to participate. Other participants may not be aware or not happy that you are observing the situation.
Quantitative Data analysisThe analysis of numeric data. This could be official data such as government statistics or the data collected by an organisation or group.Useful for understanding how policy makers use statistics to form opinions. Helps to understand the prevalence or frequency of something, such as the number of meetings held, number of members present etc.Analysis can be complex and often requires technical skills to be accurate. Data may not be accessible or may not be accurately collected.
Facilitated/Focus Group Discussions (FGDs)Qualitative group discussions that produce insights or data through a more natural conversation style than 1-1, especially about group interactions.Get a range of opinions and perspectives as well as an insight into how people interact with others.Facilitated/Focus Group Discussions (FGDs) Qualitative group discussions that produce insights or data through a more natural conversation style than 1-1, especially about group interactions. Get a range of opinions and perspectives as well as an insight into how people interact with others. Even for those used to talking about sexuality can find it difficult to raise these issues in a public forum. Discussions can be dominated by certain individuals and some perspectives may not be heard.

Conducting the Audit: Questions to consider

Once you have decided on an approach and selected which set of policies you are going to audit, you might want to consider the following questions:

  1. Who is the audience for this policy audit and what form should it take to make most impact?
  2. Do those conducting the audit have a common understanding of the terms sexuality, gender and heteronormativity?
  3. Who are the different actors who have influence in the policy process?
  4. How are people being involved in the policy process?
  5. What are the ways that different voices are included or excluded from the policy process?
  6. How do people who break gender and sexual norms engage in the process? Are decision makers making spaces for them to engage? If not, why not?
  7. What are the different forms of knowledge and types of evidence that feed into the policy process (scientific evidence, personal stories, meeting minutes, academic research etc)?
  8. How do the policies define the family unit and how are gender relations conceptualised?
  9. Whose interests are represented and reinforced by the policies?
  10. Are there ways in which people engage with the policy process that are unexpected?

Challenges

Below are some of the challenges encountered by researchers conducing policy audits on sexuality issues. It might be useful to consider these learning points in your group or team as part of the audit planning process.

  • Work on sexuality can be challenging because it is seen as a private issue and therefore can be seen as threatening or hard to discuss for certain individuals or groups. This might include individuals within your organisation.
  • Whilst key word searches are helpful, terms change over time and may not bring the desired results.
  • Be realistic about what is achievable. Identify the policies you want to investigate and the entry points you intend to use.
  • Be clear about the difference between speaking ‘on’ and ‘off’ the record to interviewees and ensure they are clear about the aims of your research and have given written consent to be included. Make sure all of your staff are aware of the importance of confidentiality, what protecting anonymity means, and how you will protect your sources.
  • Before proceeding, check how accessible the documentation that you are going to need is. Are government documents available online? Are legal documents and court proceedings available to the public? Are policy-making processes documented and the information freely available? This will guide what approach you take and could save you time.
  • With your team, make an assessment of your personal contacts. Do you know anyone in the government who could help with accessing certain documents? Are there people connected to your organisation who might be willing to be interviewed ‘off the record’? These ‘insider’ connections could be vital in getting you access to the information you need.
  • Try to look for alternative stories, for those who can reveal different realities to the ones in the policy.
  • Keep an eye on government websites and any announcements regarding the drafting of legislation. In most cases, government are obliged to notify citizens of impending legislation and provide spaces for them to give feedback. Monitoring government communication allows for the opportunity to get involved in policymaking processes and thereby gain insider access.
  • Ensure that your analysis is inclusive of all individuals who are marginalised because they break sexual and gender norms, not just LGBTI individuals.

Further Reading:

Guide to Participatory Action Research and other adaptive methods

Policy Audits