Novick e Catley. 2012

 NOVICK, Laura R.; CATLEY, Kefyn M. Assessing Students’ Understanding of Macroevolution: Concerns regarding the validity of the MUM. *International Journal of Science Education*, v. 34, n. 17, p. 2679–2703, 2012. DOI 10.1080/09500693.2012.727496.

2680

- Microevolution: "Variation within a species due to natural selection..."

- Macroevolution: "...the formation, radiation, and extinction of species and higher-order natural groups comprising a most recent common ancestor and all its descendents (e.g. Amniota) over long periods of time."

- A maior parte da instrução escolar é em microevo. Várias refs interessantes sobre isso.

- Filogenetica e cladística é a melhor forma de tratar macroevo.

2681

- Até o MUM (Measure of Understanding of Macroevolution) de Nadelson and Southerland (2010), havia somente o CINS (Anderson et al., 2012) e o ORI (Nehm; Schonfeld, 2008) que visa somente microevo). Tem o Genetics Concept Assessment também (Smith; Wood; Knight, 2008).

2682

- Sobre o MUM: Deep time; phylogenetics/classification; speciation; fossils; nature of science.

- "...the framework could be further strengthened by fleshing it out in more detail in ways that make contact with the relevant prior literature."

- Segue o padrão AERA et al (1999).

2683

- Reliability: "Reliability concerns the extent to which a test provides consistency of measurement".

- Validity: "...concers whether the test measures what it is intended to measure".

- "Reliability is a prerequisite for, but not a guarantee of, validity."

2684

- Segundo Kaplan e Saccuzzo (2009), a forma de lidar com evidência relacionada a conteúdo para validade é usar expertise informada por problemas de aprendizagem já diagnosticados (varias refs interessantes aqui) em conjunto com literatura primária.

- content-related evidence: "... the extent to which the content of the test maps onto the construct the test is intended to measure" p. 2683

- criterion-related evidence: resultados do teste comparados a outros benchmarks. p. 2683

PROBLEMAS DO PRIMEIRO CICLO

2685

- *transformation into* is very misleading.

- "... implying that the entire population of taxon A transformed wholesale into taxon B... This is not how evolution works ... such a progression is inherent in the everyday meaning of *transformation*". p. 2685-2686

- The cladogram only shows extant taxa.

2686

- *evolved into/from* > Bad wording

- Anagenis exists, but not as a model of speciation.

PROBLEMAS DO SEGUNDO CICLO

2686

- Nautillus, Murex e Octopus tratados como espécies quando são gêneros com muitas espécies.

2687

- "With respect to test items, misinterpretation reduces the validity of a test".

- Mais problemas terminologicos

- "Test takers should perform well or poorly on the MUM because of their knowledge  of macroevolutionary concepts, not because of their ability to infer the equivalence  of the pigment and simple optic cups."

2688

- Tem uma questão sobre fósseis de vertebrados no meio também, mas o ciclo é de moluscos.

PROBLEMAS DO TERCEIRO CICLO

2688

- Os diagramas filogenéticos utilizados são mal-formulados.

- "... cladograms (by definition depicting only di- and polychotomous branching) should be the phylogenetic diagrams of choice, consistent with contemporary scientific practice (Schuh e Brower, 2009)"

PROBLEMAS DO QUARTO CICLO

2688

- "Each depicted taxon is represented by multiple terminal branches. The meaning of these different branches is not specified in the figure and is unclear given contemporary understanding of the phylogeny of these taxa."

OUTROS PROBLEMAS

2689

- Datas exatas devem ser reservadas para eventos mais significativos.

- Tempo relativo é mais recomendado. Ordenar eventos, testar entendimento da relatividade temporal em uma regua arbitrária.

- "...transform them from rote memorization probes to probes of conceptual understanding of evolutionary history"

2690

- Deixar claro se os evento foram X anos *atrás* ou *por quanto tempo* eles duraram.

- Segundo Haladyna e Downing (1989), "numerical answer options for multiple-choice questions should be placed in numerical order".

- Taxa are not organisms: "The distinction between organisms and taxa is critical for understanding evolution  because taxa (not organisms) are the entities that undergo evolutionary change over time".

- "Genealogical relationships are known as tokogenetic and are distinguished from phylogenetic relationships, which are hierarchical (Hennig, 1966)."

ISSUES CONCERNING THE INCORRECT ANSWER ALTERNATIVES

2691

- "Use distractors that are plausible and avoid those that are absurd (Hadadyna; Downing, 1989, p. 40)."

- No MUM, muitos dos distractors são implausíveis em qualquer contexto, mesmo além daquele avaliado na prova, diminuindo a quantidade efetiva de itens para o estudante e aumentando a chance de acerto ao acaso.

- Fazem uns testes estatísticos com um redacted MUM e mostram que os itens tem um acerto superior ao esperado.

2697

- "the results indicate that performance was above chance for 19 (70%) of the 27 itens."

DISCUSSION

2698

- Importância da Tree of life

2699

- "If students are able to reject certain distractors as implausible regardless of the information presented, this increases the likelihood that they will answer the question correctly simply by guessing and will lead to overestimating their understanding. On the other hand, if students are able to select the correct answer without reference to the scenarios, diagrams, and question stems, then their accurate response to such items does not reflect their ability to analyze the associated information. Both of these situations, which we documented for the MUM, reduce the construct validity of the test."

- Importância do tree-thinking: boas refs

- "Accordingly, we suggest that two-part assessment, consisting of a carefully constructed concept inventory and tree-thinking evaluation, would be a powerful tool in the science educator's toolbox for evaluating students'..."