MERLOT/Physics Peer Review Criteria
The following information is provided for users of MERLOT/Physics to better understand the evaluation of MERLOT materials. This information is also meant to help new reviewers to quickly become participants in the MERLOT Peer Review process.
The three general categories, or "dimensions", for evaluation used for reviews are:
- Quality of Content
- Effectiveness as a Teaching-Learning Tool
- Ease of Use
These three categories are interdependent to some degree, but separate ratings are given for each. The "Effectiveness" rating is given with respect to one or a few specific uses of the material (e.g., Lecture/Demo, Tutorial, Homework Assignment). So, for example, an item may be highly effective as a lecture/demo with the instructor running the application, but confusing when run by students. Effectiveness ratings are for the use thought best by the reviewers.
Reviews are on a 1 to 5 scale, with 1 being very marginal (items deserving less than a 1 should be removed from MERLOT) and 5 being superior. The criteria listed below outline the characteristics of quality material in the three review dimensions. Materials need not meet all these criteria to be highly rated on MERLOT. Specifically, material that is deemed useful is given a rating of 3. Material that positively satisfies two or three of the main (numbered) questions in each of the categories below is given a rating of 4. Material that positively satisfies many of the main (numbered) questions is given a rating of 5. This list is subject to change as more experience is gained in reviewing learning materials.
Authors see all reviews before they are posted and have a chance to respond to the review or request it not be posted. There is a space to add comments from authors in the posted reviews.
QUALITY OF CONTENT:
- Does the material present valid (correct) concepts, models, and results?
a. For quantitative material, are the numerical results correct and in agreement with graphs or figures?
b. For qualitative material, are the trends correct (e.g., doubling an input doubles an output for a linear relation)?
c. Is the textual material accurate and precise?
d. Does the material follow clearly defined physics notation and conventions?
- Does the material present important physics concepts or models?
a. Is the content part of the core curriculum in physics?
b. Is the content difficult to teach and/or learn?
c. Is the content a prerequisite for more advanced material?
d. Is the content unique, not covered in standard collections of learning material?
- Does the material help develop conceptual understanding of physics?
a. Does the material use a variety of ways to present physical concepts?
b. Does the material represent an original approach to the topic covered?
c. Can students interact with the material and receive timely feedback about the impact of changing physical parameters?
- Does the material make effective use of graphics and multimedia?
a. Are the graphics attractive?
b. Do the graphics/media effectively illustrate physical quantities?
- Is the material flexible?
a. Can the material be used to explore a range of different physical situations, or is it applicable to only one or a few physical systems?
b. Is there control over the initial state of the system and other important physical parameters?
c. Is the material modular so that it can be adapted to different physics problems?
EFFECTIVENESS AS A TEACHING-LEARNING TOOL: (Use Stated)
- Does the material relate to the learner's knowledge and needs ?
a. Does the material build links to the learner's prior knowledge?
b. Is the material challenging and engaging?
c. Does the material provide an opportunity for the learner to control and direct the learning experience?
- Does the material promote knowledge development ?
a. Are the learning goals, either explicit or implicit, clear?
b. Does the material develop significant knowledge (Foundations, Applications, Integration, Human Aspects)?
c. Will the material help students retain and use this knowledge in other contexts?
- Does the material provide a quality learning experience?
a. Does the material provide dynamic and concrete visual examples?
b. Is the material flexible so that it can be used for a variety of different kinds of learning situations or experiences?
c. Does the material incorporate the elements of active learning?
d. Does the learner experience a sense of progress during the learning experience?
- Does the material provide learners with quality feedback?
a. Does the material give students frequent and immediate feedback on their performance?
b. Are the criteria clear for discriminating between superior, acceptable, and poor performance?
c. Is the feedback given in a positive and supportive manner?
EASE OF USE:
- Does the material operate in an understandable manner?
a. Does the material load and run on (a) standard computer system(s) in a manner transparent to experienced users?
b. Can the material be used by students with different levels of experience with computers and/or the web?
c. What is the expected learning curve for students and faculty? d. Will students become lost or confused while using the material?
- Is the general layout of the material consistent and intuitive?
a. Is the design attractive and inviting for students to use?
b. Do buttons, sliders, and other elements have obvious uses and work well?
c. Are the input and output elements easy to understand, operate, and read?
- Does the material provide effective feedback?
a. Is information displayed in a manner with which students will be familiar?
b. Is information displayed in a graphical manner?
c. Is feedback immediate, while users are interacting with the material?
- Is the material documented and does it have useful instructions?
a. Does the material include physical explanations, equations, or derivations?
b. Are documentation and instructions available and easy to understand?
c. Is there interactive or case-sensitive help?
d. Is technical support necessary for either instructors or students?
e. Have the authors described any technical problems with the material?