2.1 Virtual policy dialogue design
We held a three-hour virtual policy dialogue via Zoom in June 2021 as the final phase in our mixed-methods study on the capacity of the MHSUH workforce in Canada. The event was part of our integrated knowledge mobilization strategy and was designed to generate stakeholder-driven policy implications and priority actions based on our research findings. The objectives of the policy dialogue were to share the key findings of our research, assess and foster consensus regarding the policy implications of these findings, and identify priority action items to move towards evidence-informed policymaking. The format and plan for the policy dialogue were designed by the research team, the study’s pan-Canadian expert advisory committee, and an expert facilitator external to the research team, who was hired for the event (see Table 1 for policy dialogue agenda). We incorporated elements of nominal group technique, using XLeap (a digital facilitation software that enables the capture, ranking and prioritization of input from a group of participants) to generate ideas and understand prioritization. Classic nominal group technique involves silent idea generation regarding a particular question, round-robin sharing of ideas, group discussion, and ranking of ideas by individual vote (21, 22). The nominal group method differs from the Delphi method, another well-established consensus-building strategy, in its structured, small-group, face-to-face interaction; whereas, the Delphi is premised on anonymous and asynchronous input and feedback (23).
Two weeks before our virtual policy dialogue, in preparation for the event, all participants received an evidence brief of our preliminary study findings on the impact of the pandemic on the capacity of the MSHUH workforce in Canada. Following a research presentation, participants were divided into virtual breakout rooms with six or seven participants in each room. Breakout rooms were pre-assigned to ensure a mix of stakeholder sectors in each group and to allow participants to speak in their preferred language (French or English). In each breakout room, discussion was facilitated by a team member who was familiar with the research study and trained in XLeap, the facilitation software. The small groups were asked to reflect on and discuss the possible policy implications of our research findings (Focus 1). Ideas were brainstormed using XLeap and participants were able to see the ideas generated on the screen. Each small group was then asked to collate their ideas as best as possible and identify their top three policy implications.
Following the small group discussion, the external facilitator engaged the whole group in a discussion of the top three policy implications of each group, highlighting the key points, clarifying the possible policy directions, and creating a synthesized list of the ideas generated. These ideas were then individually ranked by participants using XLeap to determine which policy directions were considered the highest priority. The same process was followed to identify and prioritize key action items and next steps based on the identified policy implications (Focus 2).
With some adaptation, our virtual policy dialogue retained most features of established in-person policy dialogue practices. According to Boyko et al. (25), deliberative stakeholder dialogues have three defining features: a constructive meeting environment, a purposeful mix of participants, and the appropriate use of research evidence. Addressing a high priority issue, pre-circulating a policy brief, skilled facilitation, not attributing comments to individuals (Chatham House Rule), not emphasizing a need for consensus, and follow-up activities to support actions are also features contributing to the success of policy dialogues (17; 26). These and other elements were synthesized by Damani et al. (18) in their article on an in-person policy roundtable. Drawing on their work, we have compared our facilitated virtual policy dialogue to these key features and guiding principles in Table 2. The one significant difference was that our dialogue explicitly built toward consensus. In a virtual environment, with less time and space for generative discussion, a technology-assisted and facilitated priority setting activity helped to drive the discussion toward a more concrete outcome.
2.2 Participants
We invited participants to attend the policy dialogue based on their knowledge of and interest in the MHSUH workforce. Our research study was guided by a pan-Canadian expert advisory committee composed of knowledge users and collaborating organizations; members of this committee were invited to the policy dialogue and asked to identify further possible attendees. During our qualitative stakeholder interviews, we told participants about the policy dialogue and asked if they had suggestions of others we should invite. We directed our invitations to senior leaders in government departments and organizations, asking them to suggest delegates if they could not attend.
We prioritized inviting a diversity of stakeholder perspectives as much as was feasible, including across organizational sector, occupation, lived experience, geographic region of Canada, and ethno-racial and Indigenous identity. A total of 46 stakeholders representing a variety of sectors from across Canada attended the virtual policy dialogue (see Table 3 for a profile of stakeholder attendees). The policy dialogue followed Chatham House Rule; thus, we are not reporting any identifying information beyond the organizational sector of each participant.
2.3 Ethics
This research study was approved by the research ethics boards at the University of Ottawa and Athabasca University. Written online consent forms were provided to all participants in advance of the policy dialogue and the link was provided in the Zoom chat at the beginning of the meeting for anyone who had not yet completed the form. A recording prompt on Zoom required participants to consent to the recording to remain in the online meeting room.
2.4 Data collection and analysis
Data generated from policy dialogues come from purposeful conversations where contributors collectively create new understandings by complementing research evidence with their own experiences and knowledge on a particular issue (20). The data collected from the virtual policy dialogue included the full transcripts from each small group discussion and the main sessions, reflective field notes from the research team members who facilitated the breakout rooms, and the final report from the external facilitator. This final report captured all the ideas entered on the XLeap platform in the small groups and the synthesized and prioritized ideas from the main sessions. Audio recordings from the main session and breakout rooms were transcribed verbatim and coded using NVivo.
Studies using policy or deliberative dialogues are unique in that the interpretation of research data by the dialogue participants is, in and of itself, a source of primary data (20). We developed an initial (deductive) coding framework based on the preliminary results of our main research study. As Plamandon et al.’s (20) integrated framework for analyzing data from deliberative dialogues suggests, we also used the key messages identified as policy dialogue participants engaged with our research as further (inductive) codes for categorizing emerging data from the policy dialogue. Coded data were thematically analyzed to identify key concepts raised in the small group breakout rooms and in the plenary sessions. Coding and thematic analyses were regularly discussed and shared among the research team for feedback and consensus.