We explore three analytic methods that can be used to quantify and qualify changes in attitude and similar outcomes that may be encountered in the educational context. These methods can be used or adapted whenever the outcome of interest is change in a generally unmeasurable attribute, such as attitude. The analyses we describe focus on: (1) change in total 'attitude score'; (2) item-level changes in attitudes towards different topics; and (3) 'attitude shift' that is defined based on a qualified change algorithm. In our example data, the total-score approach gives a general index to the level of positive attitude; the item-level approach gives the median level of positive attitude and indicates items with the most positive/negative attitude (i.e., items to target in future iterations). The qualified change approach provides an objective measure of whether a shift in attitude has occurred. Each analysis is described with its advantages and disadvantages using the data from a survey of 70 preclinical first and second year medical students before and after an elective 11-week interactive seminar (22 contact hours) which introduced elements of complementary and alternative medicine (CAM) into programmed medical school training. When assessing changes that are more qualitative than quantitative, any of these methods can be employed to derive either descriptive or inferential statistics. The methods are straightforward and are appropriate when measurements are imperfect, ratings are subjective and differences are not necessarily absolute.

This article describes a 'Mastery Rubric' (MR) used to design both the curriculum and the assessments in a new two-year certificate programme intended to train physicians in clinical research skills. The MR for clinical research skills is built around a set of core research skills: critical review of literature; articulation of research objective; development of research design; development of analysis plan; implementation of the study; implementation of the analysis plan and presentation of results. Four distinct levels of performance are described for each skill: beginning, novice, competent and proficient. This rubric outlines and provides a path to mastery of the clinical research skills the certificate programme was designed and funded to target. Using the rubric to design the curriculum ensures that courses will provide instruction in key domains, promotes assessment that demonstrates development in the target skills and knowledge, and encourages reflection and cognitive self-monitoring in the students. It is a flexible, criterion-referenced definition of 'success' for students as well as the programme itself. The criteria are characterised in terms of the skills, habits of mind and organisational principles that can foster excellence in clinical research, but the approach can be generalised.