Qualitative Research Methods

Course: RES812 — Doctoral Program in Computer Science (Cybersecurity & Information Assurance) Focus Area: AI-Driven Cybersecurity Research

Overview

This course develops doctoral-level expertise in qualitative research methodology. Topics span foundational traditions, research design selection, data collection, coding and analysis, trustworthiness, observational methods, human subject ethics, and preparation of a dissertation research prospectus — all applied to the domain of AI in Cybersecurity.

Qualitative vs. Quantitative Research

Philosophical Foundations

Dimension	Qualitative	Quantitative
Paradigm	Constructivist / Interpretivist	Positivist
Reasoning	Inductive	Deductive
Reality	Subjective, socially constructed	Objective, measurable
Goal	Understand meaning and experience	Test hypotheses, measure relationships
Data	Text, interviews, observations	Numbers, statistics
Sample	Small, purposive	Large, random
Analysis	Thematic, narrative, content	Regression, ANOVA, statistical
Outcome	Rich contextual insight	Generalizable findings

When to Choose Qualitative Research

Qualitative methods are best suited when:

The topic is understudied or lacks established theory
The research requires depth over breadth — understanding why and how
Phenomena are difficult to quantify (perceptions, lived experiences, decision-making)
Flexibility is needed to explore emerging themes as the study progresses
Human factors, context, and culture are central to the research question

Application to AI in Cybersecurity

Qualitative inquiry is well-suited for cybersecurity research because:

AI-driven threat response relies heavily on human judgment and perception
Organizations differ in how they integrate AI — context is critical
Ethical and cultural dimensions of AI adoption cannot be reduced to numbers
AI cybersecurity is an emerging field — established theories are insufficient

Qualitative Research Designs

Phenomenological Design

Phenomenology explores the lived experience of individuals concerning a specific phenomenon. It seeks the "essence" of the experience across multiple participants.

Purpose: Understand how people perceive, emotionally respond to, and make meaning of a phenomenon

Process: 1. Recruit participants with direct experience of the phenomenon 2. Conduct in-depth, semi-structured interviews 3. Identify significant statements (horizontalization) 4. Cluster statements into meaning units and themes 5. Write a composite description of the essence of the experience

Application in Dissertation (AI & Cybersecurity):

"What are the lived experiences of cybersecurity professionals in responding to AI-driven threats in enterprise environments?"

A phenomenological approach captures the emotional, cognitive, and professional dimensions of how analysts interact with AI security tools — nuances that quantitative metrics cannot reveal.

Grounded Theory

Grounded theory builds new theory directly from data through iterative coding and constant comparison.

Process: - Open Coding — Break data into concepts and label them - Axial Coding — Establish relationships between codes; form categories - Selective Coding — Identify the central category; integrate into a theory - Theoretical Saturation — Stop collecting data when no new themes emerge

Best suited for: Topics where no adequate existing theory explains the phenomenon

Application: Developing a framework explaining how organizations balance AI benefits and risks in cybersecurity operations

Case Study Design

Provides rich, contextual insights into how specific organizations implement AI-driven cybersecurity strategies. Identifies factors such as risk assessment, leadership decision-making, and gap analysis across industries.

Narrative Inquiry

Analyzes individual professional stories to understand career trajectories, motivations, and meaning-making over time. Applicable when longitudinal or biographical context matters.

Data Collection Methods

In-Depth Interview

A one-on-one conversation designed to develop deep understanding of participants' thoughts, feelings, and experiences.

Characteristics: - Open-ended, probing questions - Researcher-directed but participant-centered - Audio-recorded and transcribed

Best for: Personal narratives, complex phenomena, individual perspectives

Semi-Structured Interview

A guided but flexible interview combining pre-set questions with room for exploration based on responses.

Structure: 1. Warm-up questions — build rapport, understand background 2. Core questions — central research focus 3. Closing questions — offer participants space to add context

Best for: Exploratory studies with diverse participant experiences; most common for doctoral qualitative research

Sample Interview Protocol (AI in Cybersecurity)

Warm-Up: - How many years have you worked in cybersecurity, and what roles have you held? - Describe your current responsibilities and areas of specialization.

Core Questions: - How has AI changed the way you detect and respond to cyber threats? - What challenges have you encountered when trusting AI-generated alerts? - How do you validate recommendations made by AI-driven security tools? - What ethical concerns, if any, have you observed in AI-based threat detection?

Closing: - Is there anything important about AI in cybersecurity that we haven't covered? - Who else in your field would have valuable perspectives on this topic?

Focus Groups

Guided discussions with 6–12 participants led by a moderator. The group dynamic generates insights that individual interviews may not reveal.

Best for: Exploring community norms, shared attitudes, group consensus or disagreement

Document Analysis

Review of policy documents, incident reports, organizational records, and archival materials. Particularly valuable in cybersecurity for analyzing AI governance policies and threat response protocols.

Sampling in Qualitative Research

Core Principle

Qualitative sampling prioritizes depth, richness, and relevance — not statistical representativeness. The goal is to select participants who can best inform the research question.

Sampling Strategies

Purposive Sampling (Recommended for AI Cybersecurity Dissertation) - Deliberately selects participants with targeted characteristics - Ensures participants have the knowledge and experience relevant to the study - Example: AI developers, cybersecurity analysts, IT security managers, policymakers - Enhances credibility, transferability, dependability, and confirmability

Snowball Sampling - Initial participants refer others in their professional network - Used when the target population is hard to access (niche or sensitive roles) - Example: A security expert at a government agency refers colleagues

Convenience Sampling - Selects participants based on accessibility - Useful for exploratory or pilot studies; weaker for rigorous doctoral research

Sampling Plan for Dissertation

Target participants: - Cybersecurity analysts with 3+ years in AI-integrated SOC environments - AI/ML engineers working on security applications - IT Security managers responsible for AI tool adoption - Cybersecurity educators and researchers - Ethical hackers and penetration testers using AI tools

Qualitative Data Analysis

Stages of Analysis

1. Data Familiarization Immerse yourself in transcripts, notes, and documents. Read repeatedly. Note initial impressions and recurring ideas.

2. Coding Segment data into meaningful units and assign labels. Two approaches: - Inductive (bottom-up) — Codes emerge from the data - Deductive (top-down) — Codes are guided by existing theory

3. Theme Development Group related codes into broader themes that represent overarching patterns across the dataset.

4. Theme Review Ensure themes accurately reflect the data. Merge overlapping themes. Resolve inconsistencies.

5. Defining and Naming Themes Assign clear names; define each theme's meaning within the study context.

6. Reporting Present themes with supporting quotes and connect to research questions and existing literature.

Coding Methods

Method	Description	Best For
Open Coding	Line-by-line labeling without predefined categories	Initial exploration
Axial Coding	Identifies relationships between open codes	Building categories
Thematic Coding	Groups codes into broader themes	Pattern identification
In Vivo Coding	Uses participants' own words as codes	Preserving authentic voice
Descriptive Coding	Summarizes data segments concisely	Quick categorization

Thematic Analysis vs. Grounded Theory

Dimension	Thematic Analysis	Grounded Theory
Goal	Identify patterns and themes	Develop new theory
Coding	Inductive + deductive	Purely inductive
Complexity	Flexible, accessible	Structured, iterative, intensive
Outcome	Rich thematic interpretation	New theoretical model
Best when	Interpreting existing data	No adequate theory exists

Recommended for AI Cybersecurity Research: Thematic Analysis for dissertation studies focused on professional perceptions and experiences; Grounded Theory if the goal is to build a novel cybersecurity framework from scratch.

Observational Methods

Participant-Observer Roles

Role	Description
Complete Observer	Observes without interacting
Observer-as-Participant	Primarily observes with minimal interaction
Participant-as-Observer	Participates in the group while maintaining observer stance
Complete Participant	Fully embedded; research role not disclosed

Observational Guide for AI in Cybersecurity

Central Research Question:

"How do cybersecurity professionals respond to and mitigate AI-driven security threats in an enterprise environment?"

Target Setting: Security Operations Centers (SOCs) in enterprise or government environments

Target Groups: - Security Analysts — First responders to AI-generated threat alerts - Incident Response Teams — Containment, forensics, remediation - Cybersecurity Engineers — Architects of AI-integrated defense systems - IT Security Managers — Policy, compliance, AI adoption decisions - Ethical Hackers — Adversarial simulation against AI defenses

Data Sources: - Social actions — How analysts interact with AI tools in real-time - Verbal data — Semi-structured interviews and informal conversations - Artifacts — Incident reports, alert dashboards, policy documents, training materials

Learning Points in Observational Research

1. Naturalistic Observation Enhances Validity Direct observation records actual behaviors in their natural context, overcoming the social desirability bias and recall limitations of self-reported data. Application: Observe employee responses to simulated phishing emails rather than relying on survey self-assessment.

2. Patterns and Contextual Influences Observation reveals informal structures, communication styles, and workflow realities that formal documentation does not capture.

3. Bias and Ethical Considerations Researchers must balance participation with analytical distance. Informed consent, confidentiality, and minimizing disruption are essential in sensitive organizational settings.

Trustworthiness in Qualitative Research

Lincoln and Guba's Four Criteria

Criterion	Quantitative Equivalent	What It Ensures
Credibility	Internal validity	Findings accurately reflect participants' realities
Transferability	External validity	Findings apply to other contexts via thick description
Dependability	Reliability	Study is consistent and could be repeated
Confirmability	Objectivity	Findings are shaped by participants, not researcher bias

Strategies for Ensuring Trustworthiness

Triangulation Using multiple data sources, methods, researchers, or theories to cross-check findings: - Data triangulation — Interviews + observations + documents - Methodological triangulation — Qualitative + quantitative components - Investigator triangulation — Multiple researchers analyze independently - Theory triangulation — Multiple theoretical lenses applied

Member Checking Return findings or transcripts to participants for verification. Ensures interpretations accurately reflect their intended meaning.

Reflexivity Maintain a reflexive journal documenting researcher biases, assumptions, and methodological decisions throughout the study.

Peer Debriefing Have a colleague critically review your coding, interpretations, and analytical choices to minimize blind spots.

Thick Description Provide detailed, rich descriptions of participants, settings, and context to enable readers to assess transferability.

Audit Trail Document all methodological decisions, coding changes, and analytical steps to support dependability and confirmability.

Prolonged Engagement Spend sufficient time with data and participants to develop deep, nuanced understanding.

Rigor and Reducing Bias

Sources of Bias in Qualitative Research

Researcher subjectivity — Personal worldview influences interpretation
Participant bias — Socially desirable responses; impression management
Recall bias — Inaccurate recollection of past events
Selection bias — Non-representative or convenience-driven sampling

Mitigation Strategies

Use non-leading questions; pilot test the interview guide
Apply purposive sampling with clear inclusion criteria
Maintain a reflexive journal from day one
Use direct participant quotes rather than paraphrased summaries
Apply inter-coder reliability checks — two researchers code independently, then compare
Conduct member checking after analysis
Establish anonymity to reduce social desirability bias

Human Subject Ethics

Anticipated Risks for Cybersecurity Research Participants

Confidentiality risk — Discussing AI vulnerabilities may inadvertently reveal sensitive organizational information
Professional repercussions — Participants may fear backlash from employers for critiquing AI tools or security decisions
Psychological stress — Reflecting on past security failures can cause anxiety

Risk Mitigation Strategies

Informed Consent — Full disclosure of study purpose, data handling, and voluntary participation before any data collection
Anonymization — Replace names and organizational identifiers with codes throughout all documents
Data Security — Encrypt all data; restrict access to authorized researchers only
IRB Approval — Submit full ethics protocol before beginning participant recruitment
Right to Withdraw — Participants may exit at any stage without penalty or explanation
Confidentiality Agreements — Remove or aggregate firm-specific information that could identify organizations

Computer-Aided Qualitative Data Analysis (CAQDA)

NVivo for Doctoral Research

NVivo is purpose-built for qualitative and mixed-methods research. Key capabilities:

Strengths for AI Cybersecurity Research: - AI-assisted auto-coding accelerates thematic identification across large datasets - Supports multiple data formats: text, audio, video, PDF, social media - Word frequency queries and text search identify emerging terminology patterns - Concept maps and matrix coding visualize relationships between AI attack patterns and defense strategies - Collaboration features support team-based research with full audit trails

Limitations to Consider: - Steep learning curve; plan training time before data collection begins - AI auto-coding can misinterpret technical cybersecurity context — human review is essential - Requires data pre-cleaning before import - Software license costs may be a barrier for independent researchers - Better suited for static datasets; less effective for real-time dynamic environments

When to Use CAQDA

Use NVivo or similar tools (ATLAS.ti, MAXQDA) when: - Working with large volumes of interview transcripts or policy documents - Managing data from multiple sources simultaneously - Needing audit trail documentation for doctoral committee review - Conducting team-based coding that requires version control and consistency

Research Prospectus

Purpose and Structure

The Research Prospectus is a milestone document demonstrating alignment between:

Study Problem — What practice-based gap exists in the field?
Study Purpose — What does this study aim to do, and how?
Research Question(s) — What specific question does the study answer?
Research Rationale — Why is qualitative methodology the right choice?

Study Problem Statement

The problem to be addressed by this study is the growing gap between the sophistication of AI-driven cyber threats and the ability of cybersecurity professionals to detect, understand, and respond to them in real-world enterprise environments. While AI-based security tools are increasingly deployed, there is limited research on how practitioners perceive, trust, and operationalize these tools in Security Operations Centers — particularly when AI-generated alerts conflict with human judgment or when adversarial AI techniques create novel, unpredictable attack vectors.

Research Questions (Phenomenological)

Central Question:

What are the lived experiences of cybersecurity professionals in perceiving and mitigating AI-driven security threats in enterprise Security Operations Centers?

Sub-questions:

Q1a: How do SOC analysts describe their trust in AI-generated threat alerts? Q1b: What strategies do cybersecurity professionals use to validate AI recommendations? Q1c: What ethical concerns do practitioners associate with AI-driven automated responses?

Rationale for Qualitative Approach

A qualitative phenomenological approach is most appropriate because:

AI-driven cybersecurity threats are adaptive and context-dependent — statistical models cannot fully capture their emergent nature
The study focuses on human perception, decision-making, and organizational culture — inherently non-numerical phenomena
The field lacks established theory for how practitioners navigate human–AI interaction in security operations
Semi-structured interviews with experts provide rich, first-person accounts that inform real-world practice

Course Summary and Key Takeaways

Qualitative research is the right tool for complex, emerging, human-centered problems — especially in AI cybersecurity where quantitative data exists but context and meaning are missing
Design selection matters — Phenomenology for lived experience, Grounded Theory for building new theory, Case Study for organizational context
Sampling is purposeful — Quality over quantity; purposive sampling ensures participants can genuinely inform the research question
Coding is iterative — Thematic coding is flexible and powerful; move from open codes to themes through systematic review
Trustworthiness is actively built — Triangulation, member checking, reflexivity, and thick description are not optional — they are the standard of rigor
Ethics extends beyond IRB — Confidentiality, power dynamics, and psychological safety require ongoing attention throughout the study
NVivo accelerates analysis — But researcher judgment cannot be delegated to software; human oversight of AI-generated codes is essential
The Prospectus aligns everything — Problem → Purpose → Question → Method must form a coherent, defensible chain of reasoning

References

Creswell, J. W., & Poth, C. N. (2021). Qualitative Inquiry & Research Design: Choosing Among Five Approaches. SAGE.
Creswell, J. W., & Creswell, J. D. (2014). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. SAGE.
Maxwell, J. A. (2013). Qualitative Research Design: An Interactive Approach. SAGE.
Tracy, S. J. (2024). Qualitative Research Methods. Wiley-Blackwell.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic Inquiry. SAGE.
Charmaz, K. (2014). Constructing Grounded Theory. SAGE.
Mweshi, G. K., & Sakyi, K. (2020). Application of sampling methods for the research design. Archives of Business Research, 8(11).
Donkoh, S., & Mensah, J. (2023). Application of triangulation in qualitative research. Journal of Applied Biotechnology and Bioengineering, 10(1).
Poth, C. (2023). The SAGE Handbook of Mixed Methods Research Design. SAGE.