MSc Computer Science - PROM05: Research Methods & Data types

Evaluating My Research Method & Data Collection Approach

As I continue developing my dissertation project on client-side deepfake detection, I have begun to reflect more carefully and consider the research methods and data types that will be most suitable for evaluating this work. Because my project will involve both technical system development and user experience, selecting an appropriate research design is essential to ensuring that the findings are reliable, meaningful, and relevant to real-world use.

After reviewing different research approaches, I have decided that a mixed-methods research design is the most appropriate for this project. This approach combines quantitative and qualitative methods, allowing both technical performance and user perception to be examined together. Creswell and Plano Clark (2018) explain that mixed-methods research strengthens validity by integrating multiple forms of evidence, which aligns closely with the dual nature of my project.

From a quantitative perspective, I will be collecting measurable data to evaluate the technical performance of the prototype deepfake detection tool; this will include accuracy, detection speed, false positives, and resource usage, providing an objective assessment of how well the system identifies manipulated media. I will also compare detection outcomes to understand whether certain categories of content, such as low-resolution videos or heavily altered images, are more challenging for the system to detect. In addition to this, basic benchmarking of client-side processing, including CPU load, memory usage, and latency, will help assess the practicality of the tool for real-world deployment. Structured scoring within the evaluation chain will allow aspects of system effectiveness, usability, and risk reduction to be quantified; these numerical measures will act as evidence to support any interpretation of the qualitative findings, linking system performance to real-world applicability (Sokolova and Lapalme, 2009).

From a qualitative perspective, I will explore the broader context, meaning, and implications of the prototype within social media environments. This will involve a threat landscape analysis to examine how deepfakes are currently used for fraud, harassment, political manipulation, and misinformation. I will also conduct a review of existing detection methods, comparing industry and academic approaches to understand their strengths, limitations, and why accuracy may drop in real-world scenarios. I will evaluate privacy and ethical considerations, including concerns around server-side detection, user trust, consent, and data handling. The project will incorporate a cybersecurity strategy and evaluation chain, interpreting how a client-side tool contributes to a wider defence posture. I will also assess the feasibility of a client-side/Chromium extension, exploring design trade-offs, potential use cases, limitations, and user impact.

By combining these two forms of data, I aim to develop a more comprehensive understanding of the prototype’s effectiveness. Quantitative results will indicate whether the system performs reliably, while qualitative feedback will show whether it is understandable, trustworthy, and suitable for everyday use. This integrated approach will also allow any technical findings to be interpreted within a real-world context, strengthening the overall validity of the research.

One challenge that is associated with this approach is managing time and resource limitations. Conducting extensive technical testing and large-scale user studies may not be feasible within the available timeframe. To address this, I plan to focus on smaller-scale but targeted evaluations that still provide meaningful insights. This will involve prioritising core performance measures and carefully designed questionnaires that capture key user concerns without excessive data collection.

Overall, this stage of the project has highlighted the importance of selecting appropriate research methods and data collection strategies. By adopting a mixed-methods approach supported by established academic frameworks, I aim to ensure that my project is both technically rigorous and socially relevant. This will support the development of a deepfake detection tool that is not only accurate but also practical and trustworthy for use within social media environments.

References

Creswell, J.W. and Plano Clark, V.L. (2018). Designing and Conducting Mixed Methods Research. 3rd edn. Thousand Oaks, CA: Sage.

Sokolova, M. and Lapalme, G. (2009). A systematic analysis of performance measures for classification tasks. Information Processing & Management. Available at: https://www.sciencedirect.com/science/article/pii/S0306457309000259 (Accessed 30 Jan. 26).