EXPLORING THE DESIGN AND USE OF MOLECULAR ANIMATIONS THAT CONFLICT FOR UNDERSTANDING CHEMICAL REACTIONS^#

Abstract

Understanding chemical reactions conceptually involves recognizing characteristics of observable phenomena and envisioning how atoms, ions and molecules move and interact to cause the macroscopic changes. Our research focuses on the development of effective strategies for designing and presenting visualizations (videos and animations) to assist students with making connections between macroscopic and molecular level behaviors of chemical reactions. Specifically, we study how students, who view videos of a redox reaction that exhibits obvious signs of macroscopic chemical change, can determine which molecular animation of a set of contrasting animations is best supported by its fit with experimental evidence. Herein we describe how we develop our videos and animations, and how students are learning from this animation task. Students who select inaccurate animation models are often enticed by a model that is easier to explain and fits with their understanding of reaction equations. We note that even though students indicate a preference for one animation over another, they often revise their drawn representations to fit with features from multiple animations. With the assistance of eye tracking research, we are gaining a better understanding of what students view and how they make sense of it.

Keywords:
molecular animations; visualizations; chemistry education research; first-year undergraduate; general chemistry; reduction-oxidation reactions

INTRODUCTION

Scientific models are important for representing submicroscopic entities, because students cannot observe these phenomena directly. ¹ Although models do not correspond exactly to the complex concepts they represent, they tend to bring certain features into focus while obscuring others, helping scientists to make predictions about behaviors within a system.¹ Molecular animations, a main component of our research, are models that specifically depict the movement and interactions of atoms and molecules lending insight into the submicroscopic level. They are a useful educational support that help students build mental models of the behavior of particles.² Several studies report that animations are very effective and students' have much better conceptual understanding of chemistry events after viewing them.^3-13 However, learning is often reported to be uneven, and many students continue to have difficulty conceptualizing the particulate level.^2,14-18 Transforming students learning requires providing them with experiences where they must critically self-examine the assumptions and beliefs that have structured their interpretation of the experience.¹⁹ The goal of this research has been to help students learn how to use experimental evidence, presented in videos, to evaluate the plausibility of three different molecular animations of a reaction event: One animation that is scientifically accurate and two that are inaccurate. In essence, we are suggesting a structured use of errors as a learning tool and charging the viewer with understanding what makes an unacceptable model, as well as, what makes an animation an acceptable model.

MOLECULAR ANIMATIONS

Animations hold the potential to clarify concepts that are otherwise abstract and difficult to understand to assist in the development of students' conceptual understanding. But the design of these tools and determining the key elements that distinguish a good animation from a bad one is not simple. Often instructors will critically review the information portrayed in animations deeming them inaccurate or too complicated for their students. Typically, instructors use animations to discuss certain features that are relevant to their lesson, yet they rarely ask their students to critique the animations as an instructional practice. Several studies have focused on how students make sense of what they see in animations and how this information positively enhances students' understanding of atomic level events.^2-13 Kelly noticed that students tend to incorporate aspects of animations into their observable explanations when they notice the variance between their models and the animation models.² However, she observed that when students were asked to describe how animations were similar to or different from their mental models they had difficulty and often failed to represent many details in their drawn and oral explanations of the chemistry concepts.² The students seemed to be unclear on how to justify their mental representations. Clearly, how students take in information from animations is complex.

Eye tracking can be a useful tool for gaining insight into where students focus their attention while viewing animations and how certain cues can assist or detract from the viewing experience. Hansen conducted an eye tracking experiment and observed that college General Chemistry students employed a variety of viewing patterns (e.g. focused their attention to certain areas) when they engaged with chemistry simulations that included visualizations and connections between the symbolic and submicroscopic levels.²⁰ Cluster analysis of viewing patterns when combined with analysis of verbal and drawn responses suggested that students engaged with different chemistry representations through distinct strategies. For example, some groups of participants focused on specific representations exclusively (e.g. no fixations recorded in the symbolic area of the simulation), some shifted between images (e.g. focused only on the symbolic, then only on the submicroscopic representation), and sometimes students gave no visual attention to viewing areas, because the student was not ‘looking' at the entire image.

ANIMATION COMPLEXITY/ USE OF STRUCTURED ERRORS

While animations provide an efficient way to convey the dynamic and molecular-level details of chemistry events, the level of complexity that should be depicted to most usefully model chemistry events remain uncertain. For example, should animations provide detailed, "accurate" representations of molecular events or should animations portray only the most relevant events by reducing or eliminating features considered unessential to the molecular processes being conveyed? Lowe reported that "construction of a high quality mental model from a complex visual display relies on the capacity of the learners to extract appropriate information from that display" (p 225).²¹ Ideally, animations should be designed not only to portray content (scientific or otherwise) but to accommodate the perceptual and intellectual attributes of the learner. Cognitive Load Theory (CLT) suggests that for effective learning to occur from any instructional tool, the mental demands of the task (cognitive load) must not exceed the limitations of working memory. ²² Therefore, animations should be designed to reduce the information presented to essentials whenever possible. Unfortunately, students may think they are learning a great deal, and they may even have greater confidence in their answers, but their actual learning gains may not be substantial.²³ Muller observed that when videos presented very simple and clear explanations, physics students did not engage deeply enough with the media to realize that the video events were in variance with their mental models.²³ Kelly observed that students were most influenced by the very general characteristics of animations, and they spent less attentional effort on detailed pictures, dismissing features as unimportant.² These decisions affect how students adapt what they see in the visualization to fit with their understanding. Therefore, there is a clear need to teach students the value and limitations of models.

Using structured errors as a method of learning is a relatively new approach to teaching.²⁴ Coppola and Pontrello shared a two-stage peer review and discussion technique that they used with students enrolled in Organic Chemistry to help students identify and discuss errors.²⁴ This activity was done in partnership with a website-driven approach, in which two sets of quiz solutions were posted, consisting of answers that were consistent with the course together with answers that contained instructor-generated errors and inconsistencies. Students were required to post comments to account for the solution inconsistencies to earn full credit on the quiz. They observed that students working explicitly with errors were deeply engaged and found the activities meaningful, but difficult and frustrating. However, they also noted that during the process student groups did not always converge on the same answers and grades did not significantly change. Muller noted that when students viewed a video of a student providing a wrong explanation that was corrected by another student making connections to experimental evidence, this caused many students greater confusion and reduced confidence in their understanding, even though their responses on test questions improved.²³ Our research is therefore focused on examining how working with errors depicted in animations of chemical reactions will affect student learning. Our goal is to develop learners who are able to critique visual representations based on experimental evidence, which is a skill that is needed not only by STEM professionals, but also most importantly by all scientifically literate citizens.

CHEMICAL REACTIONS

Several studies indicate that students have a variety of alternative conceptions in their understanding of chemical equations and reactions.^25-28 For instance, students who can successfully balance chemical equations often do not understand the relationship to the particulate or molecular level.²⁸ Even when students are given diagrams representing molecules before and after a reaction, they have difficulty describing the reaction and writing an equation.²⁶ In another study, Ben-Zvi et al. observed that students held a static view of reactions and lacked an understanding of the dynamic nature where bonds break, atoms rearrange, and new bonds form.²⁷ Hesse and Anderson reported similar findings where students did not understand that reactions involved the rearrangement of atoms.²⁹ In general, chemistry students often have difficulty relating symbolic equations to the particulate nature of the events. ³⁰

Chemical reactions cause students difficulty because there are multiple places for misunderstanding³¹ that can be grouped into four major areas: 1) Students fail to grasp the idea that during a chemical reaction the entities reacting are changed in fundamental ways, which means that the products no longer show the same properties as the reactants; that although the atoms involved in the reaction are conserved, the rearrangement of their electrons leads to different properties; 2) Students are confused as to the nature of chemical bonds. Bonds are viewed as entities linking atoms, thus a giant lattice of ions, in which no ions are specifically linked to other ions, is difficult to understand in terms of bonding. Weaker interactions such as those associated with the solvation of ions are simply ignored; 3) Students find it difficult to correctly identify the bonds being broken and formed in a chemical reaction; 4) Most students do not understand the idea that chemical reactions are driven by an increase in entropy in the universe.³¹ Instead they look for a causal agent for the change. Our animations are designed to focus on these problematic areas to help students make sense of the mechanisms of the reactions and the species that are involved.

CLINICAL STUDIES INVOLVING ANIMATIONS

Initially we conducted a qualitative study to investigate how students learned from contrasting molecular animations of a reduction-oxidation experiment focused on the reaction between solid copper and aqueous silver nitrate.³² The animations used in this study varied from each other in a number of ways.³³ The scientifically accurate animation, developed by Roy Tasker and VisChem was a well-known animation that showed a detailed depiction of electron exchange in which hydrated silver ions were drawn toward a copper lattice, where the ions received electrons from copper atoms in the form of an electron cloud (Figure 1). Upon being neutralized, a silver atom attached to the copper surface and the water molecules that once hydrated the silver ion left and went into solution. Meanwhile, some distance away from the neutral silver atom, a copper ion formed and was drawn into solution by water molecules that were attracted to it.

Figure 1
Image from the scientifically accurate VisChem animation showing the silver ion gaining an electron cloud from a copper atom. With permission from Roy Tasker

In contrast to this animation, our animation artist developed an animation that looked similar structurally, but it was not narrated, it was much shorter and it portrayed a simplistic, yet incorrect mechanism in which two silver nitrates delivered two silver atoms to the copper lattice and then the two nitrates attached to a copper atom and drew it into solution. The animation also began with a macroscopic view of the copper wire immersed in a clear and colorless solution in a test tube, it zoomed into the submicroscopic level and then at the end zoomed back out showing how the reaction looked at the end with the copper wire coated with a dark substance in a clear blue solution (Figure 2). This animation reflected the physical look of a single replacement reaction and we referred to it as the Physical Exchange Animation (PEA).

Figure 2
Image from the Physical Exchange Animation showing the scientifically inaccurate representation of two nitrate ions breaking free from their silver atoms upon colliding with the copper surface

Upon completing studies involving these animations currently under review, our team revised the animations so that they would look stylistically like each other apart from the reaction mechanism. The animations were designed to be two-dimensional to maintain a simplistic look. The copper lattice was made of a well-organized array of blue copper atoms covered by light yellow electron clouds. This was done so that when the copper ions were formed by transferring their electron clouds, once the cloud was removed, it revealed blue copper ions that would be drawn into solution by water molecules and this would provide a stronger connection to the evidence that the solution changes to a blue color a noted weakness from the initial study. In addition, the nitrate ions were made to look a bit duller in color to blend into the solution background, while the purple silver ions were drawn to the copper surface and when the electrons were transferred, white dots jumped as a gray cloud appeared over the silver ion to again assist students with the connection to the physical evidence and to explicitly show the electrons involved in the transfer process (Figure 3).

Figure 3
An image of the revised scientifically accurate animation of electron exchange between silver ions and copper atoms

The physical exchange animation was constructed with the new color palette used for the scientifically accurate animation, but otherwise it resembled the first version apart from adding electrons, as white dots, to show that electrons were exchanged in the formation of the silver ion and the macroscopic connection at the beginning and end was removed to maintain consistency with the other animations (Figure 4). A third incorrect animation was created because in the initial study we noticed that many students drew representations of the molecular level prior to viewing the animation that did not fit with the mechanisms in the animations. Most of these students believed that all the molecules, atoms and/or ions ended up on the surface of the copper. Thus, this third animation was constructed to investigate whether students might be more inclined to choose an animation that fit best with this common conceptual model (Figure 5). In addition to these three new animations, the video of experimental evidence was also redesigned to resemble cooking videos that are shared on social media (Figure 6). These videos focus close-up and they are often sped up to reduce time while conveying the most salient features. Redesigning this video reduced the viewing time from thirteen minutes to approximately two minutes, which is likely better for holding students' attention since the sped-up reaction allows the student to quickly see a dramatic result rather than watching it gradually happen over thirteen minutes.

Figure 4
An image of the scientifically inaccurate revised physical exchange animation

Figure 5
An image of the scientifically inaccurate animation designed based on analysis of students’ hand drawn conceptual understanding

Figure 6
The opening screen of the newly revised video of experimental evidence

In our more recent research efforts, a pilot study was conducted in the spring of 2016 with twelve students, five females and seven males, at a midsized western university in the United States. The goal was to examine how the subjects responded to four animations that were designed to look very similar, apart from their mechanisms. Two of the animations were electron exchange animations and both were scientifically accurate. One was the electron exchange animation described and shown in Figure 3, while the other animation had a different color scheme, in which the neutral copper atoms were yellow-orange in color and the copper(II) ions that went into solution were blue (Figure 7). In addition, the copper(II) ions that were extracted by water molecules were in different places on the lattice. In the animation previously described, the copper(II) was on the surface at the interface with the aqueous silver nitrate solution (Figure 3) and in the additional electron exchange animation, the extracted copper ion was nestled between only copper atoms (Figure 7). The other two animations were scientifically inaccurate animations and resembled the animations described and shown in Figures 4 and 5. The findings of the study revealed that most students (9 of 12) chose one of the two electron exchange animations as the best animation. Only three students chose the physical exchange animation and no students chose the animation in which the silver and nitrate attached to the copper surface leaving nothing in solution. The most popular reason students gave for choosing an electron exchange animation was the authentic role of water in drawing the copper(II) ion away from the surface. Interestingly, none of the students seemed to notice the electrons being exchanged nor mentioned them in their justification; however, eye tracking studies presented later reveal otherwise.

Figure 7
An image of the scientifically accurate electron exchange animation extracting a copper(II) ion from the middle of the copper lattice and with a different color scheme

Of the students who chose the inaccurate physical exchange animation, they chose it because, "you could see the trade happen" and it connected to the equation. Regardless of most students choosing the electron exchange animations, many students struggled to adapt their picture revisions to fit solely with the animation they chose. For example, students incorporated aspects of both the electron exchange and physical exchange animations as they struggled with their molecular level understanding and the symbolic representation of the chemical equation, specifically the single replacement equation. One student, Jane, whose name has been changed to protect her identity, described her feelings about her drawings (Figure 8) made prior to seeing the set of four animations:

Figure 8
Drawings made prior to seeing the set of contrasting animations. a) Jane’s drawing of copper submerged in a solution of aqueous silver nitrate made prior to the reaction and before Jane saw the animations. b) Janes drawing of the solution after the reaction

I'm not so confident about molecules, how I drew them, because I'm not familiar with that part, but I do understand the single displacement reaction, and how the conductivity stayed because of the metals... I'm least confident about the demonstration of the molecules that I drew, like how it all formed. I don't think it's as precise as it should be, but that's how I understood it, because it's simpler to understand it that way.

Jane's molecular drawings indicate that she did not distinguish the silver nitrate ions as separated ions nor in very much detail (Figure 8). It is also very challenging to understand from her drawings how she thinks a single replacement reaction is happening. After Jane viewed four different animations, she selected the version of the electron exchange animation consistent with Figure 3, but her reason reflects that she mistook the water molecules that moved the copper ion to be nitrate ions.

Because the silver displacement ...I can see how the water could be moving the silver to there (copper surface), acting like a bus to the copper and then moving the copper out. But I don't know if it's right or wrong. I can see it happening, I can make sense, but for me the third one made a clear image that the silver moved from here and then the nitrate kind of moved the copper to there, because those are the molecules that are actually making the changes.

It's important to note that the animation showed the nitrate ions to be floating about in solution, but they had nothing to do with the reaction. This is counter to what the student describes and then she realizes it.

But I feel like if that's the case, then the water is what is helping the silver move to the copper and then when the water moves into here, it moved the silver here (to the copper surface) and it attaches onto a copper, moving it from the copper nitrate, this aqueous solution.

In the end, Jane chose the most scientifically accurate animation, but her reasoning reflected that she was very attached to the reaction equation and she believed that the nitrates must be involved. However, her picture revisions revealed that she was not completely certain that the electron exchange animation was best. In fact, she tended to blend the look of both the physical exchange animation and the electron exchange animations, by drawing silver nitrate as both separated ions and as a molecular pair (Figure 9) and by showing that both a water molecule and a nitrate ion attracted to a copper ion and drew it into solution (Figure 10). It was unclear whether she recognized that copper was an ion. Through this activity, Jane has made great improvement on the molecular details she included in her drawing, and she was developing the skills to self-correct her understanding, although we can see that she was still struggling with the electron exchange mechanism and the role of water in the reaction. She seemed to be deeply invested in figuring out which animation was best.

Figure 9
Jane’s revised drawing, made after viewing the contrasting animations, of the chemical species prior to the reaction

Figure 10
Jane’s revised drawing, made after viewing the contrasting animations, of the species during the reaction

EYE TRACKING INVESTIGATION

A key aspect in the development of these visualizations is the documentation of features students direct their attention to as they thoughtfully consider what they are viewing. We have been using eye tracking studies to discern students' viewing patterns and to assist us in examining whether the depicted features in variance are focal hotspots, also known as areas of interest (AOIs). Analysis of student AOIs, combined with their verbal descriptions of the experimental videos and animations, informed subsequent redesigns of videos and animations. Semi-structured interviews gave information on why students attended to specific visual features and ignored others, and how they attended to the differences between the animations. For example, in one eye tracking study, students were presented with the redox experiment video before they were shown still images from this video and asked to describe the molecular level that resulted in the macroscopic image. The three new molecular animations were presented three times and eye tracking data was collected during all viewings. The participants were asked to consider how features in the animation were supported or refuted by the experimental evidence and eye-tracking analysis was performed (Figure 11). Eye tracking data was analyzed by defining chemically relevant aspects of the animation, for example water molecules, nitrate ions, the silver ions, and the copper lattice. Chemically equivalent features were grouped together (e.g. the water molecules surrounding a specific silver ion), because students can glean the same information by viewing any of these equivalent water molecules, thus these viewing patterns were analyzed together. Species that reacted differently were coded separately (e.g. the silver ion that deposited onto the copper solid was coded separately from the silver that remains in solution). Viewing patterns were combined with verbal descriptions, allowing the research team to identify aspects of the animations viewed, but not mentioned, by students.

Figure 11
Instantaneous snapshot of the defined dynamic areas of interest (AOIs) used in the analysis showing groupings of chemically synonymous AOIs

Eye tracking data supported the assumption that students viewed key aspects of the experimental video. In addition, animation eye tracking data revealed information about where students focused their attention and helped us discern how students responded to animated nuances such as how the students coped with the removal of water molecules that were shown to disappear. This disappearance of the water molecules was employed to avoid an overcrowded reaction environment and we learned that students understood our shortcut. During the interviews students correctly identified the water molecules as fading away to reveal the redox reaction more clearly. We also gleaned that students followed the explicit exchange of electrons between atoms and ions in the electron exchange mechanism based on students' viewing patterns as detected by the eye tracker (Figure 12). Eye tracking data confirmed that some key features of the animation (such as the electron transfer) are viewed but not mentioned by the students during the interview. This finding confirmed that the animation was designed in a manner that enables students to view the key features of the reaction.

Figure 12
A static comparison of participant viewing the electron transfer area (left, area of dynamic electron transfer circled in black) versus the solvation of copper ion by water molecules (right). Each small circle represents a single participant’s viewing at that moment in the animation

CONCLUSION

In conclusion, with instructional design demands focusing on deeper and more meaningful learning there is an increased need for materials/tools that can be used by students both independently and collaboratively. Our comprehensively studied visualizations indicate that participants in our study were very attached to the symbolic equations and they were less experienced reflecting on the molecular level. When shown animations, students pay attention to the details of what they view as confirmed from eye tracking studies and from the qualitative studies in which they described and listed the features that were observed. Interestingly, when students were shown contrasting animations, struggles often were revealed. We have reason to believe that when the animations differ in complexity, students prefer simplistic representations. The reason is simple. They can explain them and, in a world where test scores and grades are of central importance, easy-to-understand wins. When the complexity of the animations is similar, the struggle that students have in deciding which animation is best becomes more apparent and we see that even though they indicate a preference for an animation they may incorporate aspects of the animations they did not select.

The implication of this work is that we need to empower our students with opportunities to try out ideas and test models for their strengths and their limitations throughout our courses. The activity helps students develop their conceptual understanding of the molecular level as it fits with experimental evidence, and it encourages students to question their understanding to select the best animation. This animation activity would work well as a hook to engage students in the powerful connection between macroscopic evidence and the molecular level. When used in this manner, it might motivate students to be more attentive during the bulk of the lesson. The activity would also work well as a formative assessment task to check for understanding and to assess whether students can navigate between macroscopic and submicroscopic levels. Lastly, the activity could be used as a closing exercise to challenge students and stimulate deeper reflection on what they have learned in class. In short, there are many ways to make use of this animation puzzle to challenge students to critique animations for their scientific accuracy while getting students to make connections to and build on their prior knowledge.

• #
  This paper was presented in the SBQ - ACS symposium on Chemical Education, held in Goiânia, on May 2016. Publication costs were assisted by INCT Inomat, CNPq process 573644/2008-0

REFERENCES

• ¹
  National Research Council; A Framework for K-12 Science Education: Practices, Crosscutting concepts, and Core Ideas. National Academies Press, Washington D.C., 2012.
• ²
  Kelly, R. M.; J. Chem. Educ. 2014, 91, 1152.
• ³
  Kelly, R. M.; Jones, L. L.; J. Sci. Educ. Technol. 2007, 16, 413.
• ⁴
  Kelly, R. M.; Jones, L. L.; J. Chem. Educ. 2008, 85, 303.
• ⁵
  Höffler, T. N.; Leutner, D.; Learning and Instruction 2007, 17, 722.
• ⁶
  Yezierski, E. J.; Birk, J. P.; J. Chem. Educ. 2006, 83, 954.
• ⁷
  Velázquez-Marcano, A.; Williamson, V. M.; Ashkenazi, G.; Tasker, R.; Williamson, K. C.; J. Sci. Educ. Technol. 2004, 13, 315.
• ⁸
  Ardac, D.; Akaygun, S.; J. Res. Sci. Teach. 2004, 41, 317.
• ⁹
  Wu, H.; Krajcik, J.; Soloway, E.; J. Res. Sci. Teach. 2001, 38, 821.
• ¹⁰
  Sanger, M.; Phelps, A.; Fienhold, J.; J. Chem. Educ. 2000, 77 1517.
• ¹¹
  Burke, K.; Greenbowe, T.; Windschitl, M.; J. Chem. Educ. 1998, 75, 1658.
• ¹²
  Sanger, M. J.; Greenbowe, T. J.; J. Res. Sci. Teach. 1997, 34, 377.
• ¹³
  Williamson, V. M.; Abraham, M. R.; J. Res. Sci. Teach. 1995, 32, 521.
• ¹⁴
  Akaygun, S.; Jones, L.; Chem. Educ. Res. Pract. 2013, 14, 324.
• ¹⁵
  Rosenthal, D. P.; Sanger, M. J.; Chem. Educ. Res. Pract. 2012, 13, 471.
• ¹⁶
  Rosenthal, D. P.; Sanger, M. J.; Chem. Educ. Res. Pract. 2013, 14, 286.
• ¹⁷
  Tasker R.; Dalton R.; Chem. Educ. Res. Pract. 2006, 7, 141.
• ¹⁸
  Tversky, B.; Morrison, J. B.; Betrancourt, M.; International Journal of Human-Computer Studies 2002, 57, 247.
• ¹⁹
  Merriam, S. B.; Caffarella, R. S.; Baumbartner, L. M.; Learning in adulthood a comprehensive guide, John Wiley & Sons: San Francisco, 2007.
• ²⁰
  Hansen, S. J. R.; Ph.D. Thesis, Teachers College, Columbia University, 2014.
• ²¹
  Lowe, R.; European Journal of Psychology of Education 1999, XIV, 225.
• ²²
  van Merrienboer, J.; Sweller, J.; Psychol. Rev. 2005, 17, 147.
• ²³
  Muller, D. A.; Ph.D. Thesis, School of Physics, University of Sydney, Australia, 2008.
• ²⁴
  Coppola, B. P.; Pontrello, J. K.; J. Chem. Educ. 2014, 91, 2148.
• ²⁵
  Ahtee, M.; Varjola, I.; International Journal of Science Education 1998, 20, 305.
• ²⁶
  Garnett, P. J.; Garnett, P. J.; Hackling, M. W.; Studies in Science Education 1995, 25, 69.
• ²⁷
  Ben-Zvi, R.; Eylon, B.; Silberstein, J.; Educ. Chem. (London, U. K.) 1987, 24, 117.
• ²⁸
  Yarroch, W. L.; J. Res. Sci. Teach. 1985, 22, 449.
• ²⁹
  Hesse, J. J.; Anderson, C. W.; J. Res. Sci. Teach., 1992, 29, 277.
• ³⁰
  Kelly, R. M.; Barrera, J. H.; Mohamed, S. C.; J. Chem. Educ. 2010, 87, 113.
• ³¹
  Boo, H. K.; Watson, J. R.; Sci. Educ. 2001, 85, 568.
• ³²
  Kelly, R.; Akaygun, S.; Hansen, S. J. R.; Villalta-Cerdas, A.; Chem. Educ. Res. Pract (under review)
• ³³
  Kelly, R. M. In Technology and assessment strategies for improving student learning in chemistry; Schultz, M., Schmid, S., Holme, T., eds.; ACS: Washington, DC, 2016. pp. 117-136.

Publication Dates

• Publication in this collection
  May 2017

History

• Received
  05 Dec 2016
• Accepted
  07 Mar 2017