Large score differences between two administrations without a change in learning indicate weak test-retest reliability.
Good content coverage means high content validity, while inconsistent scoring by different examiners means weak inter-rater reliability.
Consistency shows reliability only; further evidence is needed to show the test measures what it is intended to measure.
Natural observation takes place in real settings without imposing predetermined conditions.
Observation at a set time and place under specified conditions is controlled observation.
A checklist records only the presence or absence of listed behaviors.
Giving the same test to the same group twice and comparing scores examines test-retest reliability.
Consistency between two equivalent forms with different items is parallel-forms reliability.
Comparing performance across two halves of one test is split-half reliability.
Judging a test by surface appearance without empirical evidence is face validity.
Comparing test scores with later performance in the program (predictive evidence) is criterion validity.
Difficult reading passages make the test also measure reading ability, so it does not purely measure mathematical reasoning (construct validity concern).
When most items come from two of ten topics, the test does not represent the content domain, threatening content validity.
Systematic score differences caused by different interpretations of the criteria reduce inter-rater reliability.
Multistructural responses have many facts; relational responses integrate them, so the difference is quantity versus integration.
An unrelated answer shows the task was not understood, which is the Prestructural level.
SOLO progresses from one idea to several disconnected ideas, then integrated relationships, then new generalization.
Consistent scores show reliability, but memorization items do not measure critical thinking, so construct validity may be lacking.
Explaining a concept and giving an example of it shows Understand (explaining and exemplifying).
Transforming information into a diagram that represents its meaning is a form of Understanding (interpreting).
Listing several facts without relating them is the Multistructural level.
Using only one relevant aspect is the Unistructural level.
Several correct but disconnected facts indicate the Multistructural level.
Showing how several ideas interact to form an integrated whole is the Relational level.
Applying an integrated understanding to a new context and generating a novel idea is the Extended Abstract level.