ALTMAP

Plan and implement purposeful initiatives

Description

ALTMAP is a framework that synthesizes components of program evaluation for planning and implementing purposeful initiatives. The components include Assessment, Logic Model, Theory of Change, Metrics Tracker, A+ Inquiry, and Project Plan.

• Assessment: Five categories of assessment (needs, theory, process, outcome, and efficiency) are important to consider when planning and implementing an initiative.

• Logic Model:  A logic model is a snapshot of an initiative, often summarized on one page. The logic model contents are informed by the assessment categories.

• Theory of Change: A theory of change is a basic visual representation of an initiative. The theory of change is informed by the assessment categories and logic model contents.

• Metrics Tracker: A metrics tracker is a tool for compiling measurable indicators identified in the logic model.

• A+ Inquiry: A+ Inquiry is a framework for operationalizing metrics identified in the metrics tracker as needed.

• Project Plan: A project plan is a tool for monitoring implementation of activities identified in the logic model. 

Assessment

Five categories of assessment are important to consider when planning, implementing, and conducting a comprehensive evaluation of a program. The categories include needs, theory, process, outcome, and efficiency, which are based on types of assessment questions posed by Rossi et al. (2004). Needs assessments are intended to improve understanding of needs relevant to a context. Theory assessments are intended to improve understanding of whether strategies and activities that are required for implementation of a program are appropriate to help meet the identified needs. Process assessments are intended to improve understanding of whether the program is implemented as planned. Outcome assessments are intended to improve understanding of whether expected outcomes of the program are achieved. Efficiency assessments are intended to improve understanding of whether the program is implemented with efficient utilization of resources. The image below depicts basic reflection prompts for each assessment category.

Logic Model

A logic model is often presented as a one-page snapshot of a program. Elements of a logic model typically include the context and need underlying a program, program title, expected goal, target population, resources or inputs, activities, outputs, short-term outcomes, mid-term outcomes, and long-term outcomes (Kekahio et al., 2014; W.K. Kellogg Foundation, 2004; Wyatt Knowlton & Phillips, 2013). The resources or inputs element represents what is required to develop and implement a program. The activities element represents what is intended to be done with the resources or inputs in order to develop and implement a program. The outputs and outcomes elements represent intended results of program development and implementation. The image below depicts an example of a logic model template.

Theory of Change

A theory of change  is a basic visual representation of how change is expected to occur through a program (Dhillon and Vaca, 2018, Wyatt Knowlton and Phillips, 2013). Theories of change may include causal relationships that build upon logic models. The structure of a theory of change is likely to be unique for each project. The  image below depicts a blank example of how a theory of change may appear with a variety of shapes representing key elements and intended results of a project as well as arrows demonstrating relationships between the elements and results.

Metrics Tracker

A metrics tracker is utilized as a tool for compiling values representing measurable outputs and outcomes. A metrics tracker may include fields to indicate whether each metric represents an output or an outcome, to identify the output or outcome category that aligns with the metric, and to track the target and actual values of each metric. The image below depicts an example of a metrics tracker template.

A+ Inquiry

The A+ Inquiry framework is utilized as needed to operationalize the output and outcome metrics that are tracked in the metrics spreadsheet and to help ensure they serve a relevant purpose. A+ Inquiry synthesizes common stages of disciplined inquiry processes, such as research, assessment, and evaluation (Anderson et al., 2014). The model intends to promote purposeful data collection and the alleviation of mindless decision-making and data hoarding (i.e., the collection of data that are never used). A+ Inquiry may be utilized for a variety of purposes, such as planning and implementing an inquiry process, establishing a common language for discussing assessment and other data use processes, providing rationale for decisions and actions, and diagnosing gaps in inquiry processes. The framework is displayed as a circle with eight stages connected by a hub. Each stage and the hub begin with the letter A. The stages include Absorb, Ask, Accumulate, Access, Analyze, Answer, Announce, and Apply, which are bound in the center by a hub of Awareness. The image below depicts the A+ Inquiry framework. Additional details about the framework are available HERE.

Project Plan

A project plan is utilized as a tool for tracking progress toward the completion of the intended activities and smaller tasks within each activity category. A project plan may include fields to track the start date, end date, status, and notes relevant to the implementation of each activity category and task. The image below depicts an example of a project plan template.

Synthesis of Frameworks and Tools

The relationships between the frameworks and tools are identified by assessment category. For example, the red “Needs assessment” category aligns with the “Need/Context” description in a logic model.

The yellow “Theory assessment” category aligns with the “Activities” column of a logic model are appropriate to meet the identified need. The descriptions in the “Activities” column of the logic model align with the yellow activities elements in a theory of change if included. Activities represented in the logic model and theory of change, along with granular corresponding tasks, are entered into a project plan spreadsheet.

The green “Process assessment” category aligns with the “Outputs” column of a logic model. The descriptions in the “Outputs” columns of the logic model align with the green output elements in a theory of change if included. Metrics relevant to outputs are entered into a metrics spreadsheet. The A+ Inquiry framework may be utilized as needed to help ensure the output metrics are important to track and possible to generate.

The blue “Outcome assessment” category aligns with the “Outcomes” columns of a logic model. The descriptions in the “Outcomes” columns of the logic model align with the blue outcome elements in a theory of change if included. Metrics relevant to outcomes are entered into a metrics spreadsheet. The A+ Inquiry framework may be utilized as needed to help ensure the outcome metrics are important to track and possible to generate.

The orange “Efficiency assessment” category aligns with the resources in the “Resources/Inputs” column of a logic model. 

Resources

• Anderson, N. (2022). Synthesizing frameworks and tools to develop a plan for evaluating an online data utilization curriculum for teachers. Evaluation and Program Planning, 94. https://doi.org/10.1016/j.evalprogplan.2022.102148

• Anderson, N., Ova, A., Oliver, W., Greer, D., & Carson, E. (2020). Evaluating the Develop Your Data Mindset online curriculum for educators: An assessment of needs theory, processes and outcomes. North Dakota Department of Public Instruction, Information Technology Department, Statewide Longitudinal Data System. http://dx.doi.org/10.13140/RG.2.2.34557.95206