Introduction

Document analysis is a form of qualitative research in which documents are interpreted by the researcher to give voice and meaning around an assessment topic (Bowen, 2009). Analyzing documents incorporates coding content into themes similar to how focus group or interview transcripts are analyzed (Bowen,2009). A rubric can also be used to grade or score document. There are three primary types of documents (O’Leary, 2014):

• Public Records: The official, ongoing records of an organization’s activities. Examples include student transcripts, mission statements, annual reports, policy manuals, student handbooks, strategic plans, and syllabi.
• Personal Documents: First-person accounts of an individual’s actions, experiences, and beliefs. Examples include calendars, e-mails, scrapbooks, blogs, Facebook posts, duty logs, incident reports, reflections/journals, and newspapers.
• Physical Evidence: Physical objects found within the study setting (often called artifacts). Examples include flyers, posters, agendas, handbooks, and training materials.

Rationale

Document analysis is a social research method and is an important research tool in its own right, and is an invaluable part of most schemes of triangulation, the combination of methodologies in the study of the same phenomenon (Bowen, 2009). In order to seek convergence and corroboration, qualitative researchers usually use at least two resources through using different data sources and methods. The purpose of triangulating is to provide a confluence of evidence that breeds credibility (Bowen, 2009). Corroborating findings across data sets can reduce the impact of potential bias by examining information collected through different methods. Also, combining qualitative and quantitative sometimes included in document analysis called mixed-methods studies.  

Process

Before actual document analysis takes place, the researcher must go through a detailed planning process in order to ensure reliable results. O’Leary outlines an 8-step planning process that should take place not just in document analysis, but all textual analysis (2014):

1. Create a list of texts to explore (e.g., population, samples, respondents, participants).
2. Consider how texts will be accessed with attention to linguistic or cultural barriers.
3. Acknowledge and address biases.
4. Develop appropriate skills for research.
5. Consider strategies for ensuring credibility.
6. Know the data one is searching for.
7. Consider ethical issues (e.g., confidential documents).
8. Have a backup plan.

A researcher can use a huge plethora of texts for research, although by far the most common is likely to be the use of written documents (O’Leary, 2014). There is the question of how many documents the researcher should gather. Bowen suggests that a wide array of documents is better, although the question should be more about quality of the document rather than quantity (Bowen, 2009). O’Leary also introduces two major issues to consider when beginning document analysis. The first is the issue of bias, both in the author or creator of the document, and the researcher as well (2014). The researcher must consider the subjectivity of the author and also the personal biases he or she may be bringing to the research. Bowen adds that the researcher must evaluate the original purpose of the document, such as the target audience (2009). He or she should also consider whether the author was a firsthand witness or used secondhand sources. Also important is determining whether the document was solicited, edited, and/or anonymous (Bowen, 2009). O’Leary’s second major issue is the “unwitting” evidence, or latent content, of the document. Latent content refers to the style, tone, agenda, facts or opinions that exist in the document. This is a key first step that the researcher must keep in mind (O’Leary, 2014). Bowen adds that documents should be assessed for their completeness; in other words, how selective or comprehensive their data is (2009). Also of paramount importance when evaluating documents is not to consider the data as “necessarily precise, accurate, or complete recordings of events that have occurred” (Bowen, 2009, p. 33). These issues are summed up in another eight-step process offered by O’Leary (2014):

1. Gather relevant texts.
2. Develop an organization and management scheme.
3. Make copies of the originals for annotation.
4. Asses authenticity of documents.
5. Explore document’s agenda, biases.
6. Explore background information (e.g., tone, style, purpose).
7. Ask questions about document (e.g., Who produced it? Why? When? Type of data?).
8. Explore content.

Step eight refers to the process of exploring the “witting” evidence, or the actual content of the documents, and O’Leary gives two major techniques for accomplishing this (2014). One is the interview technique. In this case, the researcher treats the document like a respondent or informant that provides the researcher with relevant information (O’Leary, 2014). The researcher “asks” questions then highlights the answer within the text. The other technique is noting occurrences, or content analysis, where the researcher quantifies the use of particular words, phrases and concepts (O’Leary, 2014). Essentially, the researcher determines what is being searched for, then documents and organizes the frequency and amount of occurrences within the document. The information is then organized into what is “related to central questions of the research” (Bowen, 2009, p. 32). Bowen notes that some experts object to this kind of analysis, saying that it obscures the interpretive process in the case of interview transcriptions (Bowen, 2009). However, Bowen reminds us that documents include a wide variety of types, and content analysis can be very useful for painting a broad, overall picture (2009). According to Bowen (2009), content analysis, then, is used as a “first-pass document review” (p. 32) that can provide the researcher a means of identifying meaningful and relevant passages.

In addition to content analysis, Bowen also notes thematic analysis, which can be considered a form of pattern recognition with the document’s data (2009). This analysis takes emerging themes and makes them into categories used for further analysis, making it a useful practice for grounded theory. It includes careful, focused reading and re-reading of data, as well as coding and category construction (Bowen, 2009). The emerging codes and themes may also serve to “integrate data gathered by different methods” (Bowen, 2009, p. 32). Bowen sums up the overall concept of document analysis as a process of “evaluating documents in such a way that empirical knowledge is produced and understanding is developed” (2009, p. 33). It is not just a process of lining up a collection of excerpts that convey whatever the researcher desires. The researcher must maintain a high level of objectivity and sensitivity in order for the document analysis results to be credible and valid (Bowen, 2009).

The Advantages of Document Analysis

There are many reasons why researchers choose to use document analysis. Firstly, document analysis is an efficient and effective way of gathering data because documents are manageable and practical resources. Documents are commonplace and come in a variety of forms, making documents a very accessible and reliable source of data. Obtaining and analysing documents is often far more cost efficient and time efficient than conducting your own research or experiments (Bowen, 2009). Also, documents are stable, “non-reactive” data sources, meaning that they can be read and reviewed multiple times and remain unchanged by the researcher’s influence or research process (Bowen, 2009, p. 31).

Document analysis is often used because of the many different ways it can support and strengthen research. Document analysis can be used in many different fields of research, as either a primary method of data collection or as a compliment to other methods. Documents can provide supplementary research data, making document analysis a useful and beneficial method for most research. Documents can provide background information and broad coverage of data, and are therefore helpful in contextualizing one’s research within its subject or field (Bowen, 2009). Documents can also contain data that no longer can be observed, provide details that informants have forgotten, and can track change and development. Document analysis can also point to questions that need to be asked or to situations that need to be observed, making the use of document analysis a way to ensure your research is critical and comprehensive (Bowen, 2009).

Concerns to Keep in Mind When Using Document Analysis

The disadvantages of using document analysis are not so much limitations as they are potential concerns to be aware of before choosing the method or when using it. An initial concern to consider is that documents are not created with data research agendas and therefore require some investigative skills. A document will not perfectly provide all of the necessary information required to answer your research questions. Some documents may only provide a small amount of useful data or sometimes none at all. Other documents may be incomplete, or their data may be inaccurate or inconsistent. Sometimes there are gaps or sparseness of documents, leading to more searching or reliance on additional documents then planned (Bowen, 2009). Also, some documents may not be available or easily accessible. For these reasons, it is important to evaluate the quality of your documents and to be prepared to encounter some challenges or gaps when employing document analysis.

Another concern to be aware of before beginning document analysis, and to keep in mind during, is the potential presence of biases, both in a document and from the researcher. Both Bowen and O’Leary state that it is important to thoroughly evaluate and investigate the subjectivity of documents and your understanding of their data in order to preserve the credibility of your research (2009; 2014).

The reason that the issues surrounding document analysis are concerns and not disadvantages is that they can be easily avoided by having a clear process that incorporates evaluative steps and measures, as previously mentioned above and exemplified by O’Leary’s two eight-step processes. As long as a researcher begins document analysis knowing what the method entails and has a clear process planned, the advantages of document analysis are likely to far outweigh the amount of issues that may arise.

References:

Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative Research Journal, 9(2), 27-40. doi:10.3316/QRJ0902027
O’Leary, Z. (2014). The essential guide to doing your research project (2nd ed.). Thousand Oaks, CA: SAGE Publications, Inc.