Focus areas of specialization
Culturally Specific Assessment
Evaluation findings often rely on evidence obtained through the use of standardized assessment instruments.
Measurement disjuncture (Sul, 2019) refers to the misalignment that occurs when elements of an instrument development process from one worldview are applied to the instrument development process of another worldview. Measurement disjuncture affects the establishment of measurement validity, and, hence, the inferences made based on the scores derived from such assessments. As a result of measurement disjuncture, researchers are less likely to acknowledge that educational programming has had an impact when, in fact, it may have (Type II error).
Culturally specific assessment (Sul, 2019) is grounded in the research of Ladson-Billings (1995), Freire (1970), Stake (1973), and Hood (1998) and is defined as assessment that:
- Supports the academic development of students (Ladson-Billings);
- Inclusive of a willingness to nurture and support cultural competence (Ladson-Billings);
- Aims to support the development of a sociopolitical or critical consciousness (Ladson-Billings, Freire);
- Focused on constructs and measures of importance to educational practitioners and other key stakeholders (Stake); and
- Functions within a system of knowledge that exists within a named worldview (Sul, 2019).
Culturally specific assessments are offered as a possible solution to the problem of measurement disjuncture.
Language Revitalization
Modern Indigenous language revitalization efforts within North America arose from the voices of Indigenous people seeking to preserve their language and culture. With support from academic scholars concerned about the loss of Indigenous languages (Krauss, 1998), revitalization efforts have become broad and expansive (Reyhner and Lockard, Eds., 2009). These efforts revolve around the teaching and learning of these endangered languages and seek to address the continued consequences of colonization, the effects of the residential school system, and other inequities of history.
Many Indigenous-language assessments revolve around the Western language constructs of listening, reading, writing, and speaking. This approach differs from the sociocultural perspective on language assessments that offer a richer, more substantive approach to assessing the learning of Indigenous languages. Assessments are sought that address the dimensions of language-learning that include, broadly speaking, communication with cultural intention and addresses, among other elements, culturally relevant listening and speaking. An important goal for this research is a transition away from the existing structure that is grounded in the Common European Framework of Reference for Languages (CEFR) and a move towards a localized assessment that focuses on the needs for language revitalization of the sponsoring organization.
Culturally Responsive Assessment
Culturally responsive assessment follows a decades-long tradition of research that began with an exploration of teachers who were having a positive impact on their African American students. Ladson-Billings (1995) conducted a significant qualitative study on the teaching methods of teachers who consistently demonstrated academic success with African American students. Her work around this theme and classification of teaching methods launched the movement towards the acknowledgement and identification of culturally relevant pedagogy. In a culturally relevant classroom, a child’s culture is not only acknowledged but seen as a source of strength that can be utilized to attain academic success. Combining the ideas of Ladson-Billings (1995), Freire (1970), and Stake (1973), Hood (1998) proposed the following elements of a culturally-responsive assessment-development process:
- Supports the academic development of students (Ladson-Billings)
- Inclusive of a willingness to nurture and support cultural competence (Ladson-Billings)
- Aims to support the development of a sociopolitical or critical consciousness (Ladson-Billings, Freire)
- Focused on constructs and measures of importance to educational practitioners and other key stakeholders (Stake)
Development of Performance-based Assessments using Item Response Theory (IRT)
Many assessment developers seek the projection of ordinal-level performance ratings onto an interval-level measurement scale that can be used both to identify the student performance level and growth in the latent construct that is the focus of the assessment. Further, the belief that the various components of the latent construct are seen as interacting should be accommodated by the measurement model.
The multidimensional random coefficients multinomial logit model (MRCMLM; Adams, Wilson, & Wang, 1997) is applicable when multiple latent constructs (e.g., listening and speaking) are present within a single overarching latent construct (e.g., communication). The MRCMLM is grounded in the 1-parameter logistic (1PL) Item Response Theory (IRT) model, commonly referred to as the Rasch Model (Rasch, 1960). The MRCMLM is specified as a between-item multidimensional partial credit model (PCM), which is an extension of the unidimensional PCM (Masters, 1982). The PCM itself allows for varied numbers of developmental levels across measures that are grouped together within knowledge domains.
Critical Quantitative Methodologies
With a strong foundation in educational statistics, data analysis, survey design, and data structures combined with a strong background in the teaching of advanced mathematics and statistics, all applied through a critical perspective, we support our clients’ understanding and development of measurement tools and evaluation designs.
Additional Specializations
Our work focuses on educational programs and initiatives that seek a more grounded understanding of their processes and outcomes. Our specializations are many and include the following:
- Adult Education
- Capacity Building
- Critical Quantitative Methods
- Culturally Responsive Assessment
- Culturally Specific Assessment
- Curriculum Design
- Curriculum Development
- Data Analysis
- Data Systems
- Database Design
- Development of Performance-based Assessments using Item Response Theory
- Educational Leadership
- Educational Technology
- Grant Writing
- Higher Education
- Instructional Design
- Language Revitalization
- Nonprofit Organizations
- Program Design
- Program Evaluation
- Qualitative Research
- Quantitative Research
- Research Designs
- Research Methods
- SPSS
- Staff Development
- Statistics
- Strategic Advising
- Strategic Planning
- Survey Design
- Survey Research
- Teacher Training
- Technology Needs Analysis
- Technology Integration