STEPS to Making Fair and Equitable Decisions
1. Articulate comprehensive, precise, transparent, and standardized processes for establishing the minimally-qualified applicant pool and selecting the semi-finalists by answering these questions:
- Will committee members screen/evaluate/rank applicants individually or together?
- Will all committee members screen/evaluate/rank all or a staggered subset of applications?
- Will all committee members assess all or a staggered subset of materials?
- How will committee members’ ratings be aggregated?
- How will data from committee members be aggregated into a single rating for each applicant?
- By what process will the committee come to a decision about its short list? Will members vote, for example, or work to achieve consensus?
- How will committee members define and then handle potential conflicts of interest, such as an internal candidate or a prior relationship with an applicant or with an applicant’s adviser?
- How will committee communicate with applicants and department at each stage of the process?
With large applicant pools, stagger the screening and evaluating of applicants across search committee members so that each committee member reviews a subset of applications, with overlap of at least two members for all applications. For example, if a three-member committee is reviewing 30 applications, the first person reviews Applications 1-20, the second person reviews Applications 11-30, and the third person reviews Applications 1-10 and 21-30. A similar approach can be used for reviewing materials. Research/scholarship could be evaluated with one person reviewing the cover letter and CV, a second person reviewing the CV and research statement, and a third person reviewing the cover letter and research statement. Staggered approaches can be more efficient and reduce bias by encouraging full review of relevant applicant documents. Pairings of committee members with dossiers should be varied to a reasonable extent so that you don’t inadvertently create “mini-committees of two.”
2. Before application deadline, establish appropriate screening criteria for selecting minimally qualified applicants:
- Use specified required qualifications as the primary (ideally only) screening criteria.
- If using criteria not specified in advertisement, critically justify their necessity.
- Ensure that the screening criteria are applied consistently across all applicants – no exceptions.
- Wait until the priority deadline before reading any applications, and to organize applications by some method other than order of arrival.
Resources: Sample screening rubric
Establishing the evaluation process plan and setting criteria helps ensure the advertisement and evaluation criteria correspond to each other. This strategy also reduces the likelihood that the selected criteria unfairly favor specific applicants.
3. Develop effective rubrics for selecting semi-finalist and finalists before the application deadline:
- Review evaluation rubrics from previous search cycle(s).
- Establish weighting for each area of evaluation (e.g., scholarship, teaching, and diversity/ inclusion). For example, will teaching and scholarship be equally or differently weighted?
- Develop separate rubrics for selecting semi-finalists and finalists that reflect KSAOs and predetermined materials, and the weightings of each material.
- Decide what kind of scale will be used for ratings (e.g., High, Medium, and Low or 1 to 5).
- Ensure the evaluation criteria is clear and understood similarly by all committee members21,28.
- Avoid proxy measures such as citation count27 and prestige of institution or letter writers.
- Ensure candidates’ completion date of their degree is taken into consideration.
Resources: Sample semi-finalist and finalist rubrics
Avoid relying heavily on reference letters. Reference letters often reflect cultural biases in how men and women are described29. Descriptions of men often include terminology or behavioral examples that emphasize scholarship and indicate competence and natural ability whereas descriptions of women tend to emphasize teaching and include terminology and behaviors that are associated with warmth or hard work. This gendered language can activate unconscious biases in readers that affect their evaluations of job applicants. We recommend only reading reference letters after the semi-finalist list is established and limiting their use to identifying “red flags".
4. To select semi-finalists and finalists, systematically and thoroughly review dossiers, aiming for consistency across applicants:
- Review materials in the predetermined order for all candidates.
- Pilot test rubric with a few applications and make necessary adjustments.
- Ensure that all applicants and materials are independently reviewed by more than one person.
- Spend sufficient time evaluating each applicant (at least 20 minutes) and avoid distractions (bias in evaluations is increased when the raters are distracted and under time pressure)30.
- Use all and only the agreed-upon materials and activities to rate all semi-finalists’ KSAOs, ensuring the same materials are used with the same weighting and reviewed in the same order.
- Avoid “backing into criteria” by changing or adding new evaluation criteria after reading applications that are impressive but don’t fit with the rubric.
- Periodically evaluate your criteria to monitor the usefulness of assessment criteria and determine whether criteria need to be added or adjusted.
- Periodically evaluate your criteria and the list of excluded and included candidates to monitor whether underrepresented group members are being disproportionately excluded.
- Committee members should come to meetings prepared to discuss and justify the scores they assigned and deliberate the relative merits of specific applicants.
- Avoid including a single “token” woman or underrepresented minority in your semi-finalist list31.
- When applicable, openly discuss the challenge of maintaining fairness, collegiality, and confidentiality when internal applicants are part of the pool.
- Imagine removing all identifying information from every application and judging each candidate based upon your assessment of the work.
Q: Why should separate rubrics be used for each stage of review?
A: Some criteria or materials might be irrelevant or less relevant to all stages.
Q: What should be done if after the review has begun, a committee member proposes adding new criteria?
A: The committee discusses the feasibility of the addition, ideally without any reference to a specific application.
Q: What should rubrics include?
A: They should include KSAOs with weighting, rating criteria, and ideally materials to be used for evaluating KSAOs.
5. Develop systematic processes for reference checks and preliminary interviews:
- Establish the goals of reference check and preliminary interviews and develop questions based on those goals.
- Ask the same reference questions of each referee in the same order, at the same time in the search process. Ideally, the same committee member(s) should conduct all reference checks.
- Aim for interview consistency – use same communication platform, questions, order, and same interviewer(s) across the semi-finalists at the same point in the search process.
- Verify all questions comply with federal and state hiring laws and university policies.
- Avoid offering “courtesy” interviews to internal or other applicants who do not meet stated criteria and treat internal candidates the same as external candidates.
- Follow your pre-established processes regarding number of semi-finalists and adding new semi-finalists when necessary.
Resources: Sample reference check questions and template; Sample interview questions
A candidate might bring interesting strengths or attributes to the department other than those originally sought. If such cases appear, it is advisable to reevaluate and possibly modify the review criteria. It is also advisable to periodically evaluate your criteria and their implementation. Are you consistently relying on the criteria developed for the position? Are you inadvertently relying on unwritten or unrecognized criteria? Are you inadvertently, but systematically, screening out women or underrepresented minorities?
6. Aim for consistency in campus visit experience and evaluation across candidates:
- Before first candidate visits campus, create an interview process and evaluation plan.
- Provide all finalists with a detailed itinerary, as far (and equally) in advance as possible. To ensure equitable treatment, all itineraries should be similar.
- Provide finalists with list of relevant centers, facilities, or other campus resources that they might wish to access before or during their visit.
- Select interview questions that correspond with the KSAO you are evaluating. For each interview during the campus visit, ask candidates the same interview questions in the same order, with the same person asking the same questions for all candidates.
- Maintain clear and open communication with all finalists, and be honest about expectations, as well as about issues of funding, space, or other resources.
- For all finalists, explain the department’s and the university’s expectations about teaching, research, service, and the promotion and tenure process.
Resources: Sample interview questions; Planning the campus visit; Assessing interactions with students; Accessibility and accommodation resources for campus interviewees; Faculty support resources
7. Develop a standardized process for generating the final hiring recommendation.
- Decide what information you will use to make recommendation (i.e., will all candidates be considered equal at the finalist stage and be evaluated based solely on the campus visit or will candidates be evaluated holistically using application materials the preliminary interview?).
- Develop a feedback form anchored to KSAOs in your rubric to guide input solicited from faculty outside the hiring committee.
- Determine how department faculty’s input will be used in evaluating/ranking applicants and when their input will be solicited (see FAM 652.1).
- Establish the process the committee will use to make final assessments and recommendations.
- Aim for consistency in the recommendations to the department and the dean devoting equal space in the recommendation for each candidate in each area being evaluated.
- First compare candidates to the established criteria in your rubrics then use that information to compare them to each other.
- Beware of gendered language in the recommendations16.
Resources: Sample feedback forms for faculty outside the search committee; Checklist for writing recommendations; Gender-language decoder
8. Be aware of assumptions and biases that can affect search processes and decisions:
- Learn about unconscious biases and how they affect decision-making.
- As a committee, discuss assumptions about applicants or recruitment and acknowledge beliefs about groups that might influence decisions.
- Recognize that many common beliefs about underrepresented groups are unsubstantiated.
- Take DEPTH training to learn about biases and the best practices that reduce them32.
Resources: UW-Madison’s unconscious bias video or UCLA’s implicit bias series; Harver: 13 common hiring biases to watch out for; Beyond Bias
Assumptions and biases are common and can be unintentional, intentional, innate, or learned. Assumptions about groups or individuals might result in discounting or not considering specific recruitment sources or groups of applicants34. Biased decisions are most likely to occur when people presume they are objective and fail to acknowledge their biases35.