The IELA E-Learning Blog

Category: Uncategorized

Valuing Values: An E-Learning Case Study in Educational Ethics

by Evelyn Levine

“Promotion or Retention: A Dilemma in Educational Ethics”

Illumina Interactive, Inc. and Harvard Graduate School of Education

2021 International E-Learning Award Winner, Academic Division

The Dilemma: Promotion or Retention?

A decision regarding an underachieving student such as the one we see in the screen shot above—whether to hold her back, promote her, or propose an alternate school environment—is critical for the future of that student, and can be a very difficult decision for an educator to make.

These types of decisions must be made with the most information available on the student and with input from others in the school environment.

Still, even with lots of information and input, ethical dilemmas remain. Should a student be given extra leeway if they come from a challenging home environment? Should one take into consideration how other students may react to such a decision or how the decision would reflect on the school as a whole?

Answering these types of ethical questions requires a self-awareness of one’s own values and priorities and a way to grapple with competing values.

Teaching Educational Ethics

How to teach educational ethics has become a challenge in its own right. The Harvard Graduate School of Education and custom e-learning studio Illumina Interactive partnered to meet that challenge by designing an engaging, interactive, multimedia case study and then creating an online course around this case study. The goal of this course is to help users better understand the concepts, data and skills used during ethical decision-making.

Learners taking this course become engaged with the student in the case by sorting and then revisiting a list of ethical values (such as Care, Community, Justice, and Integrity). After becoming familiar with the details of a student’s case, learners are asked to make interim and then final decisions about how to handle the situation based on these values as well as the information and diverse views presented to them in the course. 

The keys for learners are to understand what informed their decision, what questions still remain for them, and how their values evolved as they progressed through the case study.

Using this e-learning program in a blended format can then result in rich discussions in break-out sessions, especially among those of differing values. 

This course is currently being taught at Harvard to both undergraduate and graduate students. It is also being used by professors teaching educational ethics courses at other universities, and in professional development sessions with educators and school and district administrators.

Sample Screens

Case materials include: 

Ada’s Report card

Support Team Report – Ada’s academic strengths and weaknesses and behavioral information

An example of a report written by Ada and a listing of her own goals

Values to rank:

Accountability, Achievement, Care, Community, Consistency, Effort, Equality, Fairness, Integrity, and Justice.

Promotion or Retention: Working through the program

I went through the program choosing the role of Guidance Counselor (other possible roles were that of Teacher and School Administrator). These three groups of professionals comprised the team tasked with recommending retention or promotion for Ada, an underperforming eighth-grade student at Innovation Academy.

I was asked to rank—in importance to me—a list of ten values (see Values screenshot) and I got an opportunity to revisit my rankings after reviewing additional information.

I got a peek into Ada’s locker which, notably, included a picture of a recently deceased brother—a victim of gun violence. I also got to hear her in her own voice talking about school; her likes and dislikes.

The case materials I reviewed were both objective and subjective and included Ada’s report card, her support team report and her own essay and goals (see Case Materials screenshot). Feedback from other members of the eighth-grade team was also made available, both individually and through an extended group discussion, along with their faculty bios.

I got a full and rich picture of Ada, as a student and a person, as she struggles in a challenging environment. And, I had a difficult decision to make.

In the end, it was a judgment call with no right or wrong answer. And, most important was not what my decision was but how and why I had made it. And, understanding my own ethical decision-making process, I would be more confident of making future such decisions and more prepared and eager to discuss this decision with others with similar and dissimilar views.

About the author: Evelyn Levine works as a Training and Staff Development Director for the U.S. Courts. She writes on worldwide learning and development trends in public and private sectors. She can be reached at

Want to Understand How a Vector Works? Now, There’s an App for That!

by Evelyn Levine

Spatial intelligence is the capacity to visualize in one’s mind the positions of objects, their shapes, their relations in space to one another and the movement they make to form new spatial relations. This ability to perform spatial reasoning in one’s own head is essential for mastering abstract ideas in STEM (Science, Technology, Engineering and Mathematics) as well as in many artistic areas and other fields.

Research shows that the ability to perform spatial reasoning can be significantly improved with practice and training. So: Could there be an educational tool developed that would truly mirror the real experience?

That was the impetus for an innovative idea by Professor Gerd Krizek of the University of Applied Sciences Technikum Vienna’s Department of Applied Mathematics and Physics to use Augmented Reality (AR) to help people learn to be better spatial reasoners.

The result is the app Vector AR3, the winner of the Mobile Learning category in the 2020 International E-Learning Awards, Academic Division.

Using augmented reality, the Vector AR3 App provides an interactive experience whereby real-world objects are enhanced by accurately rendered computer-generated 3D objects. The app has been designed around the experience of a virtual vector moving in one’s real-world physical space.

The Project

This mobile app, available on iOS and Android mobile phones and tablets, allows for a learner to manually manipulate a virtual vector in their real-life environment, as seen from the eye of the learner’s mobile camera. This “hands-on” experience helps learners “see” how vectors act in a 3D space from various angles, using the app’s guided manipulation tools and programmed calculations. The primary audience for this app is graduate students in engineering studies, though it is also appropriate for undergraduate or even high school level math and science courses.

The tool is designed to teach students how vectors work by bringing an abstract concept (literally) into their own hands. The learn-by-doing, self-directed instructional tool presents information in a step-by-step sequence. It is problem-oriented, allowing students to create and manipulate all types of vectors. This tool also promotes peer-to-peer learning, as students can work together on a shared tablet.

A key feature of this interactive, “hands-on” AR tool is that the learner does not need to visualize a vector in their mind’s eye, or even imagine how a virtual reality (VR) vector moves in space—instead, they can actually interact with a 3D vector in their own real-life environment.

The app is designed to be used by students as a self-directed tool and by university-level and secondary school-level instructors as a teaching tool. A companion instructor tutorial with suggested teaching scenarios is currently in the works. 

Screenshot of the Vector AR3 app showing how learners can manipulate vectors and surfaces in 3D space using AR.

The Promise

The decision to create an Augmented Reality (AR) app was a deliberate choice and a challenge; Professor Krizek also views his decision as an invitation to other learning professionals and innovators to follow suit.

Professor Krizek purposely developed the app as an open educational resource (OER) so that it is available to anyone at no charge. He is also hoping that the app’s success—so far, based on positive feedback from users, the app has been very successful—inspires others to use AR to develop other learning tools, particularly in scientific fields. The Vector AR3 app is available to anyone who is interested; it is free (with no advertisements), in English, and compatible with most common operating systems. 

I have found that the best way to truly understand this “hands-on” tool is by trying it out. You can download the app for free at:

Vector AR3 by FHTW by SoftVR GmbH

Inspiring Transformation in a Time of Crisis and Change

by Evelyn Levine

Professor Leire Nuere Salgado is not afraid of change. In fact, during the uncertainty of the pandemic, she spearheaded a community of educators from Universidad Francisco de Vitoria’s faculties of psychology, education, online learning, digital transformation and innovation, along with experts in the Canvas LMS and in graphic design. The team’s goal was not only to prepare for what lay ahead, but to fundamentally change the way they teach students at the university, which is located in Madrid and also known as UFV-Madrid.

Program Inception

The result was “Formar para Transformar” (“Educate to Transform”),  the 2020 IELA Award Winner for e-learning in the Academic Division. The project was designed to do nothing less than teach faculty to better prepare students to go out into the world and transform it into a better place for everyone.

As Professor Salgado explains:

“It was the last two months before summer vacations. We faced an uncertain future, not knowing if teaching would have to be online, hybrid, or if we could return to classroom teaching, Spain was in lockdown since March 2020, everyone was trying to adapt, and instruction had to go online quickly; all this produced great fatigue in both students and teachers.”

“We did not want to stop at just meeting the technological need, which was the most pressing at the time. We wanted, as one of our maxims says, “to go beyond”…it was a great moment to make our mission even truer, to deepen the relationship with the student as a method to produce the necessary impact that will lead them to transform the world.”

Program Design                     

The online program for faculty was divided into three parts; each with three synchronous sessions, employing 25 different learning methodologies and tools.

The variety of methodologies applied was innovative; the program employed a range of techniques, from standard lectures to a more active gamification method.  Student evaluation was multi-level:individual, by peers and community feedback, and via self-reflection and questionnaires. Training for faculty was provided on UFV-Madrid’s new LMS, Canvas.

The objectives of the faculty training included:

  • Inspiring the mission of transformation.
  • Instituting new pedagogical approaches and teaching objectives, new types of relationships with students, and new evaluation practices – all in a newly designed virtual classroom using the Canvas platform.
  • Focusing on impact through self-diagnosis, surveys and measurement.
This sample screenshot illustrates the formative journey based on the UFV pedagogical model, Educate to Transform.

Community Development

A standout aspect of the program is the communal nature of both its development and implementation.

To prepare faculty there were three communities comprised of four teams each:

1. The Community of Designers included experts in pedagogy, e-learning, technology and graphic design. 

2. The Community of Happeners and Facilitators. Happeners mentored the faculty members and assisted with questions on Canvas. The facilitators trained faculty on the different methodologies and evaluation systems. 

3. The Community of Superhappeners was in charge of preparing, guiding and accompanying the happeners and facilitators, and reflecting on the course’s progress and the effectiveness of its synchronous meetings. 

Inspiration from Around the World

Faculty members participating in the program were inspired to incorporate the best practices from various countries that are considered leaders and innovators in education.

Program Feedback

Course evaluations showed:

The most highly rated items by the faculty who participated in the program were those referring to the quality of the content, the attention received, and the opportunity to reflect on one’s own teaching work. The least-valued items related to the time organization of the course, as a number of teachers stated that the course included too much content for the time available.

Student evaluations of UFV-Madrid courses have improved in all areas compared to the previous year. The overall average score went from 4.77 out of 6 in the 2019-2020 academic year up to 4.84 out of 6 in 2020-2021, after faculty had completed the “Formar para Transformar” program. During such difficult pandemic times, an increase in student satisfaction is notable. That is the quantitative data available at this time. It is also worth considering for future analysis how such a multi-level systemic culture change may take a while to be accepted, appreciated and to manifest results in an organization.

UFV-Madrid’s “Formar para Transformar”  program shows that sometimes a crisis can turn into an opportunity—and even more than that, it can become an inspirational journey. 

About the author: Evelyn Levine works as a Training and Staff Development Director for the U.S. Courts. She writes on worldwide learning and development trends in public and private sectors. She can be reached at

Teaching Teachers to Teach Online

by Evelyn Levine

I spoke with Matthea Marquart, Assistant Dean, Online Education, Columbia University School of Social Work (CSSW), about the recent project she spearheaded to help instructors meet the teaching challenges of the pandemic. This project, Webinar series to support faculty who are new to teaching onlinewas named the 2020 IELA Award Winner in the Blended Learning Division.  

As instructors around the world scrambled to adjust to a new reality in March of 2020, the Columbia School of Social Work (CSSW) began preparing webinars on the mechanics and techniques of teaching online, particular for an audience of instructors who were new to online teaching. By the end of March, CSSW’s new blended program was already in place with a live webinar series and an online library of program resources.

How did they do it?

Several factors played a role in allowing the CSSW team to produce their webinar series and resource library so quickly, during a difficult time:

They were highly motivated: Being a social worker herself, Matthea felt a keen duty to serve others during the pandemic. She was able to leverage her many years of experience and skills coordinating the CSSW Master of Science in Social Work online program, which had already been in existence for 5 years, as well as her experience coordinating the CSSW development team.

They had a vision: Even in the midst of chaos and uncertainty, CSSW held fast to its own mission of social justice and educational equity by hosting this webinar series online as a free program to be offered to any instructor, anywhere, offering much-needed guidance for those new to online teaching. 

They were prepared: The course materials had already been developed as part of CSSW’s online Pedagogy Institute, and the team had a long-standing cadre of presenters who had already taught both online and in person.

Despite being developed so rapidly, out of necessity, the program succeeded in maintaining the highest standards of quality.  

The Live Webinars

Beyond covering the basics of online instruction, such as the use of a computer, webcam, and microphone, the webinar series also focused on instructor presence and student engagement. Online discussion revolved around best practices for sharing classroom materials, judicious use of the chat function, and setting the online class norms with flexibility and compassion.

The webinar series included sessions on techniques for engaging students in live online sessions, including the use of chat, polling, and breakout rooms. Tips to help keep students engaged outside of the webinars themselves included the use of announcements in Learning Management Systems, communicating with students while in the process of grading their work in case there are any pandemic-related issues to take into consideration, and encouraging students to reach out to their instructors via email and phone as needed.

Due to the nature of the situation and the professional expertise of the course developers, this blended learning program incorporated elements of the following social work models: 

Trauma-Informed Model

There was a recognition that all instructors and students were experiencing a trauma. The program included techniques of self-care and of resilience using the trauma-informed language of choice and empowerment.

Inclusive Community Model

There was a recognition that instructors may be uncomfortable dealing with both overt and more subtle discrimination in the new online instruction format.  The program offered ways to be more inclusive. It also offered guidance for instructors on how best to deal with any difficult “micro-aggressive” situations and how to find “teachable moments” in this new environment.

The Resource Library

The blended course included a robust online repository of webinar recordings and related course materials. This on-demand resource model was chosen to meet the unique demands of the moment.

How successful was the program?

Due to the critical need and also to its high quality, this program was a greater-than-anticipated success. It reached a wide audience of instructors around the globe, at all teaching levels from kindergarten to graduate school.

Some participant feedback which best summarizes the way the program met the instructors’ needs:

 “I walked away feeling empowered and supported… The webinars provided great comfort and validation to me and I did not feel alone anymore during these challenging times.”  (Haoning Richter, CPA, CIA, CCEP)

“The webinars hosted…were such a lifesaver…! I had never taught online and was not at all comfortable with the idea. I didn’t even know where to begin…” (Madeline Y. Lee, MSSW, PhD)

Over 750 instructors registered for the live webinars. To date, views of the webinar recordings number to close to 5,000. The online CSSW resource library at remains an open resource for all.

The CSSW Development Team

(Additionally, there were six volunteer presenters: Dawn Shedrick, Beth Counselman Carpenter, Zuleka Henderson, Rob Hartley, Melanie Lowe Hoffman, and Eri Noguchi.)

About the author: Evelyn Levine works as a Training and Staff Development Director for the U.S. Courts. She writes on worldwide learning and development trends in public and private sectors. She can be reached at

Teaching People to “Drive with Care”

by Evelyn Levine

The e-learning course Driving the Courtesy Van with CARE won the 2020 International E-Learning Award for the best product in the e-learning category. The course was developed by EnVision Performance Solutions, which is based in Boston, Massachusetts. I recently spoke with Irene Stern Frielich, President of EnVision, about the development of this course, as well as with the course’s instructional designer, Alison McIsaac, who walked me through a part of the program.  

Irene Stern Frielich, President of EnVision

This program provides instruction to both new and experienced van drivers for a service company on how best to achieve the four key goals in the CARE paradigm. The acronym CARE stands for:

Customer Focus




On a macro level, employees taking the course are taught to view themselves as ambassadors for their company and to recognize that their behavior is always a reflection of the company as a whole. More specifically, they are taught how to approach each customer interaction with empathy and a sense of responsibility. The course features five vignettes of challenging customer service scenarios along with Q&A and best practices discussions with a virtual coaching component in the form of an avatar. The e-learning experience is immersive and experiential and succeeds in its learning goals due to the reality of the situations presented, the focus on the customer, and the coaching component.

Authentic Storytelling

The situations presented in the course were based on real-life scenarios. They were created in consultation with members of the service company’s taskforce and are based on actual situations which have occurred. There is a genuine feel to the situations and to the characters in these situations such as realistic anxious and the angry customers. The course’s engaging vignettes allow the user to recognize actual emotions, such as annoyance, defensiveness, and anger, which they can practice overcoming in order to deal with customers in a professional manner.

Each vignette tells a story, and the stories have a purpose. They allow learners to navigate realistic situations with recognizable characters in a safe and controlled environment where knee-jerk reactions and mistakes can be re-considered for more skillful responses either immediately or on a second try.

Customer Focus

The goal of a van driver taking this course is to deliver exceptional customer service. This requires the ability to see each situation from the perspective of the customer. The course’s vignettes are designed to clue the user in to the customer’s backstory, such as realizing that possibly the customer has arrived at the moment of interaction having just been through a chaotic situation. Working with such a customer requires responding with empathy.

The customer is also not a blank slate; each has their unique personality traits, styles and preferences. Van drivers need to act with respect and responsiveness.

Also, some general rules are taught such as knowing to avoid certain topics, such as politics, in conversations with customers.


A coaching component is an integral part of this learning experience. The virtual coach, who provides feedback and discusses best practices, is based on an actual person in the company who is a highly visible and well-regarded HR professionalThe coach avatar was even made to look and sound like the real person. This creates a natural level of respect and appreciation from the employees, and further motivates them to incorporate the feedback and strive to reach the 5-star goal of the program.

In sum, this e-learning program was developed in very close collaboration with the client company, resulting in a learning experience which feels real and engaging to the intended users; and which should ultimately help them to respond with greater CARE while on the job. Plus this is clearly a more efficient and less expensive means of training drivers in these skills than an in-person approach would be given the many van drivers who need the training.

About the author: Evelyn Levine works as a Training and Staff Development Director for the U.S. Courts. She writes on worldwide learning and development trends in public and private sectors. She can be reached at

Parlez Away with Babbel’s Language Learning Podcasts

by Evelyn Levine

“Babbel Language Learning Podcasts” won a 2020 international E-Learning Award for the best product in the “mobile learning” category.

These podcasts were developed to be used either as stand-alone products or as fully integrated parts of the Babbel language learning website application. For the millions of Babbel subscribers, the podcasts provide an additional learning resource; for those who are not already Babbel subscribers, they serve as is an easily-accessible introduction to Babbel’s products. I recently spoke with Gianluca Pedrotti, the host and creator of Babbel’s language learning podcasts, who is now working from his homemade recording studio (see photos) with the podcast’s co-hosts and guests working in separate locations. He seems to be flourishing in this challenging environment due to his resourcefulness, good humor and compassion, even offering learners extra learning support at this time. Gianluca reports that during the COVID-19 pandemic, the number of podcast listeners has increased, and Babbel is reporting greater engagement in the podcasts.

The User Experience 
The podcasts are free and available to anyone, are offered in a variety of languages, and are about 10-30 minutes each in length. The key modality is listening. Learners are not asked to memorize or repeat words or to focus attention on grammatical rules. During each episode, podcast listeners hear a coach instruct a learner who is part of the podcast.

When I sampled one French podcast, I found it works very well as a highly engaging, standalone, mobile educational tool. Below are some key areas in which I thought the podcast excelled.

Listening to a Story 
I began listening to a podcast for newcomers to French, entitled Parlez Away, and found myself getting engrossed in the program as Ted from Wisconsin tries to learn conversational French in what seems like an attempt to win over his French girlfriend’s family and friends on an upcoming trip to Paris. He struggles with tricky pronunciations and formal vs. informal greetings, and learns how to introduce himself to friends and strangers and to ask and answer common conversational questions.

The conversation is authentic, with speakers who are engaging. As podcast listeners, we can easily relate to the earnest and struggling learner as the coach corrects and encourages Ted in a supportive and friendly manner. There is also a dynamic text transcript available in some of the podcasts for those who prefer to have a written version.

Immersion in a new culture
The podcasts are designed to immerse listeners in French culture and slang as we become not just learners but “insiders” into this new world. There are also a lot of cultural references to French habits, holidays, fashion, and even flirting. 

The focus of each podcast is always on what a learner needs to know to succeed in the described situation. The learning focuses on the art of conversation: what you need to know to socialize in a variety of real-life situations. It feels genuine and highly relatable, and motivates a learner to want to be able to gain the language skills to feel comfortable in these situations. I found it infinitely more engaging than the old-school “Where is the bathroom?” types of scenarios. 

The podcast coach is a key part of the experience. The coach’s method includes correcting grammar and pronunciation with an informal and friendly tone and manner. I found that listening to the coach work with Ted, the learner, gave me much-needed confidence and motivation in my attempt to master a new language.              

The podcast is available anytime, anywhere, hands-free, making the learning accessible and fun. Listen while driving, hiking, walking or waiting in line. Or when just relaxing at home. As the Babbel site advertises: “Learn French while you kick back and listen.”

It’s all about l’experience 
All in all, it was the engaging experience that brought me back again and again to listen and repeat the podcasts. I was sharing the frustration and excitement with the learner, happy to take instruction from a friendly French native, and enjoying immersing myself in new culture. And, of course, learning French!

About the author: Evelyn Levine works as a Training and Staff Development Director for the U.S. Courts. She writes on worldwide learning and development trends in public and private sectors. She can be reached at

Re-imagining Learning During a Critical Time: An ICELW 2020 Panel Discussion

by Evelyn Levine

At the recent ICELW 2020, the 13th International Conference on E-Learning in the Workplace, which was held online an international panel of experts, chaired by ICELW Conference Chair Dr. David Guralnick, discussed e-learning during this tumultuous time. The panelists were:

Imogen Casebourne, Oxford University Department of Education, Oxford, England, UK
Veronica Chehtman, Agua y Saneamientos Argentinos SA (AySA), Buenos Aires, Argentina
Iina Paarma, United Nations Development Programme, Copenhagen, Denmark
Antonella Poce, Roma Tre University, Rome, Italy

The panel chair and participants, spanning five countries, and with wide-ranging experience in both academia and industry, spoke of their many challenges. Remote work became the norm in a blink of the eye. All workplace learning immediately went online. Everything happened without plan or precedent.

The panel’s wide-ranging aspirational discussion focused on some key areas that came into sharp focus as the crisis unfolded and which can point towards fruitful areas of future development. We invite you to watch the full video of the ICELW panel here. Key areas that were discussed include:

Critical Skills

To quickly adapt to the new reality, some skills proved invaluable, most notably:

Digital skills 

There is now unanimous agreement that the digital workplace is here to stay. Not so surprisingly, the crisis highlighted that there was a widespread need to expand these skills. For some, there is a mismatch of skills, and for others a lack of necessary skills to fully operate in a digital-only environment. We need to acknowledge the fear of technology and support users to experiment with it.

Social skills 

The human skills of collaboration, communication, and empathy became and will continue to be vital to navigate the new workplace dynamics and norms and to work successfully with others who are not physically present.


Skill in managing disruption and chaos is a key to success and will be prized as never before as organizations realize that unplanned change can happen in an instant.

Critical thinking

As change accelerates, the need for critical thinking to fully understand what is happening and to find solutions takes on even greater importance.

Changing Culture 

The sudden crisis required an ongoing balancing of conflicting needs of an organization – that of allowing flexibility with the need to maintain control. 

Organizations undertook rapid e-learning development and implementation. To meet organizational goals, immersive learning as well as learning which is more in-context and just-in-time were needed.  Many predict an increase in blended learning solutions as forced learning experiments with a captive audience will allow for more comfort with using technology to reach goals even in normal times.

Human habits are hard to change, especially in such an abrupt manner as happened during the crisis. Normal change management protocols gave way to a seat-of-the-pants improvisation. Also true is that behavioral change, and a change mindset, can be advanced at a time of turbulence and urgent need.

Workplaces did and will continue to give learners permission to engage in learning activities which will be leveraged to improve work performance. Workers will see themselves as life-long learners.

For instructor-led activities, the role of the teachers expanded and will continue to expand as they experiment with new technology. Instructional technicians are becoming a prevalent and critical resource to help with this change process. As new e-learning champions arise to meet the demands of the moment, their efforts need to be acknowledged, supported and sustained.

Learning communities expanded and will further expand to allow critical expertise to be more easily shared.         

Leveraging Technology

People are asking “Where would we be now without technology?,” an acknowledgment of just how critical technology has been during this crisis.

But “Where exactly are we with it?” should also be acknowledged. Technology is a tool and only as good as how it is utilized.

Some things that work: Build the technology around people and organizational goals to ensure it is more integrated into their world. Take into account issues of attention span and current distractions. Finally, make it a positive experience for users.

The focus should always be on the goals and on matching available technology with those goals. Also on designing technology that is more user friendly and feels more natural.

Once good solutions are found, technology allows organizations to scale up efficiently and effectively.

What Now and What’s Next?

As I write this, we are approximately 6 months into this critical time.

Now is when we can begin to make assessments of what worked well and what didn’t, take inventory of technology utilized and how it performed and was integrated, and be mindful of what additional supports and resources may be needed to nurture and expand skills.

It is also a time to prepare for the future. We need to focus on the future of work and how it may – or may not – change permanently due to this crisis. This critical time may present unprecedented opportunities to expand e-learning to meet personal and organizational goals.  

More on all that coming here in future blog posts. 


ICELW Panel Participants

Panel Chair: David Guralnick, Kaleidoscope Learning, New York, New York, United States

Panelists: Imogen Casebourne, Oxford University Department of Education, Oxford, England, UK; Veronica Chehtman, Agua y Saneamientos Argentines SA (AySA), Buenos Aires, Argentina; Iina Paarma, United Nations Development Programme, Copenhagen, Denmark; Antonella Poce, Roma Tre University, Rome, Italy.

About the author: Evelyn Levine works as a Training and Staff Development Director for the U.S. Courts. She writes on worldwide learning and development trends in public and private sectors. She can be reached at

Veronica Chehtman Discusses the Current Urgency and Future Opportunities in E-Learning

by Evelyn Levine

Veronica Chehtman is currently the Learning Technologies Manager at AySA. Previously, she has worked with an array of organizations, including AVON, Banco Hipotecario Banco ProvinciaCablevisiónExxonMobilDIRECTV, Mercer, and The Clorox Company, among others. She has also consulted for NGOs and government organizations (FAO-FODEPAL, IPPDH, CIDH, Buenos Aires City Government, Secretaría de Cultura de la Nación Argentina, Ministerio de Justicia de la Nación Argentina); and served as distance education advisor for several Universities.

Veronica now teaches postgraduate programs in Learning Theories (at the Universidad Tecnológica Nacional) and Instructional Design (at the Universidad de Belgrano) and has previously taught at FLACSO Argentina, Universidad de Buenos Aires, and Universidad de Quilmes. She recently appeared as a panelist on a panel entitled Reimagining Learning during a Critical Time at ICELW 2020, the 13th International Conference on E-Learning in the Workplace. 

I spoke with Veronica about her work and her role; a slightly-edited transcript of the interview appears below.

Q: What is your current role, and how has it changed during this current crisis period?

A: I am the Learning Technologies Manager at AySA (Agua y Saneamientos Argentinos, or the Argentine Water and Sanitation Agency). My role didn’t change in terms of its functions due to the pandemic, but changed in its urgency and priorities. We had to switch to remote learning overnight. Alliances with key agents in the organizations (particularly IT and Internal Communications) had to be strengthened. Decisions had to be made on content, technology and, most important, on how to support the organization in its urgent mission.

Q: What were some key challenges and accomplishments in your work during this period? Do you foresee any long-term positive outcomes that may arise from this crisis?

A: In the context of the Covid-19 pandemic, Argentina entered lockdown and a very restrictive isolation at the beginning of March. Since then, we have begun to navigate uncertainty and huge organizational change. The vast majority of us work from home. Training had to go remote. The most difficult challenge during this period is without any doubt to generate cultural and habit changes. And doing it fast. Our best practice for eLearning used to be avoiding “big bang” implementations and always designing a gradual strategy, in which change management, awareness and taking time for generating commitment has a very important role. Now, it had to be done right away as the challenge was huge and the deadline was yesterday.

In just a few weeks, we:

  • Generated a methodology for online synchronous training, with an important new feature of close support for trainers and trainees;
  • Developed  a new coaching approach for help desk operations; and
  • Established new processes to ensure learning has occurred.

I think all this has contributed to creating “Resilient” L&D teams: Teams that learn to be agile and empathetic, and can cope with complexity and uncertainty.

Q: What does “Re-imagining Learning during a Critical Time” mean for you?

A: Re-imagining learning means aligning all this innovation to leverage post-crisis business strategy. This crisis has taught us that learning can be not only a continuous and permanent but also ubiquitous process. Moreover, that learning is a leading process to keep the workforce connected and committed in a workplace that will never be the same.

After having gone 100% digital, I visualize the future of learning as a process completely merged with every worker’s daily routines, supported by audiovisual languages, synchronous methodologies, and performance support approaches.

Q: Do you think that the process of re-imagining the future of e-learning has already begun?

A: No. Business will change, work will never be the same. But this critical time is a mostly “hands on” period. We are working very much as an ER: triage, assess, clinical evaluation and quick delivery.

Though many new ideas and thoughts about “the day after” have emerged, I think the sense of urgency calls for a “parking lot” strategy—to wait for the crisis to end and the time to analyze these new ideas.

Q: What are some next steps for the e-learning field?

A: I think the “new normal” will require companies (depending on size, industry, etc.) to develop different strategies for keeping learning ubiquitous as much as possible. I doubt that in big metropolises such as Buenos Aires, for example, commuting for training will continue being the rule.

The first step would be to seriously assess which activities must go back to face to face. Then continuous and seamless learning environments including face to face, blended and distance learning has to be considered. Workers will expect to find it anywhere they are. This means responsive platforms for mobile devices, flipped classrooms, training-the-trainer for meaningful online teaching, relevant content and useful strategies and methodologies.  

Evelyn Levine works as a Training and Staff Development Director for the U.S. Courts. She writes on worldwide learning and development trends in public and private sectors. She can be reached at

Educational Data Mining Improves Students’ Lives

by Evelyn Levine

Dr. Ryan Baker is an Associate Professor at the University of Pennsylvania and the Director of the Penn Center for Learning Analytics. His research on Learning Analytics spans the fields of Educational Data Mining and Human-Computer Interactions. Dr. Baker will be giving a keynote speech at the ICELW 2020, the 13th annual International Conference on E-Learning in the Workplace, to be held online from June 10-12.  More information about the conference, along with a registration page, is at

I spoke with Ryan about his work; a slightly-edited transcript of the interview is below.

Q: You study student engagement/disengagement while utilizing educational software. But, how, exactly do you do that?

A: We can identify if a student is disengaged by looking at what they do within learning technology. For example, are they gaming the system—asking for hints over and over just to get the answer—or systematically guessing until they get it right (e.g., A,B,C,D… A,B,C,D… A,B,C,D…, etc.) Are they becoming careless—responding quickly and making dumb mistakes? Are they using a fake user account to harvest answers for their main account? And so on. My lab uses a combination of human expertise and data mining to identify these behaviors. First we have experts help us find cases where the behaviors we are looking for are occurring, either by going out to classrooms and taking field notes on handheld apps, or by reviewing system log data. Then we build our models based solely on system log data to identify those behaviors, verifying that our models work on expert labels of entirely new data. At this point, our models are as accurate as first-line medical diagnostics.

Q: Have you studied differences in engagement across age groups and/or different cultures?

A: We have! Actually, the most disengaged population we have ever seen is medical residents learning to diagnose cancer. That’s kind of horrifying, but also unsurprising—medical residents are really busy, and have a lot of other things to think about beyond ongoing computer-based training. We’ve studied disengagement from preschoolers to middle school and high school students to undergraduates, medical residents, military cadets, and adults taking massive online open courses. Our work has involved learners from over 100 countries, with particularly in-depth work in the USA, Philippines, Mexico, India, China, the UAE, the UK, Brazil, Costa Rica, Belgium, and most recently Norway. We find that disengaged behaviors are relatively similar between countries, but the emotions and motivations that underlie the behaviors can differ a lot.

Q: Does your research help to identify and support students who are struggling and/or are at-risk? 

A: Absolutely! Many of the behaviors we identify have now been shown to predict student success over a decade later. Our work with BrightBytes, for instance, identifies students who are at risk of dropping out of high school more than 15 percentage points better than earlier methods (see our peer-reviewed paper in the Proceedings of the International Conference on Educational Data Mining) and combines those predictions with indicators of what factors were important to that prediction—e.g.,  two students might both be at risk, but one has slipping grades whereas the other has increasing absenteeism. 

Q: What’s your advice to a non-technical educator interested in obtaining and using this type of data?

A: If you aren’t technical yourself, you don’t need to learn to do data mining yourself. Find a learning system or predictive analytics solution that meets your needs and let them provide you with the data. Platforms like BrightBytes, Civitas, ASSISTments, Cognitive Tutor, ALEKS, and many others offer comprehensive solutions that you may find useful.

I’m a big believer in combining predictive analytics on student success with reports for instructors, chief learning officers, guidance counselors, etc. By combining data mining’s ability to make prediction in complicated situations with human judgment and expertise in how to support learners, I think we can make a real difference in people’s lives. 

Q: What aspect of your research do you find most exciting and what aspect is most frustrating to you?

A: I love the opportunity to learn things about learners that might have the impact to improve their lives. Even after 100 years of educational research, there’s still so much we don’t know, and we are at the forefront of discovering which behaviors hurt students more than we think (and conversely, which approaches—like taking a quick break—can be much better than we thought). The most frustrating aspect of my research is  the slow path that it sometimes takes between a new scientific finding and testing it in broader instructional practice. We need to do better at coupling scientific research in education with everyday practice!

For more information on the technical aspects of Dr. Baker’s work, please refer to some of his research papers listed below:

Educational Data Mining and Learning Analytics

Comparing machine learning to knowledge engineering for student behavior modelling: A case study in gaming the system

Population validity for Educational Data Mining models: A case study in affect detection

Evelyn Levine works as a Training and Staff Development Director for the U.S. Courts. She writes on worldwide learning and development trends in public and private sectors. She can be reached at

Professor Antonella Poce Teaches Us How to Think

by Evelyn Levine

I recently spoke with Professor Antonella Poce, an Associate Professor (and qualified as a Full Professor), in Experimental Pedagogy at Roma Tre University in Rome who has extensively studied Critical Thinking (CT) in depth. Professor Poce sees CT skills as essential, especially for educators and especially at this time. The relevance of CT skills spans the realm from worldwide politics to our individual work lives, from the arts and sciences to cultural awareness. But these skills are often not taught, or not taught well; CT is considered an essential yet unaddressed area—so essential and unaddressed, in fact, that Professor Poce calls it an “emergency” that needs to be addressed. But, before we can address the lack of education in Critical Thinking skills, we need to explore how they can be clearly understood and measured. 

The following is an edited Q&A with Professor Poce on her extensive work on Critical Thinking. 

Q: What exactly is Critical Thinking?

A: Critical Thinking is a way of approaching situations and includes asking others or oneself the following questions:

  • Am I considering this problem in an unbiased manner?
  • Am I considering all the evidence?
  • Are my arguments formulated in a clear way?

There are both general and specific domains of Critical Thinking. The general domain includes thought about societal topics such as education and health issues. The specific domain refers to a specialized area of knowledge such as engineering, the humanities, biology, etc. 

Q: How do we assess Critical Thinking?

A: Essays, open-ended questions and analysis of collaboration, among others. I envision Artificial Intelligence (AI) supporting human assessments in the near future.

Q: How do new technologies relate to Critical Thinking?

A: Technology is not the enemy of thought. When it is used appropriately, technology can support the 4Cs: Collaboration, Communication, Critical Thinking, and Creativity. I call this type of usage  the “critical use of technology.” Technology assists me in my daily work through international communication and collaboration. Additionally, a recent Museum Art project found that using technology for student collaboration improved Critical Thinking among university graduate students.

Q: Why is this topic so important at this time?

A: Critical thinking development is a contemporary emergency that should be addressed with care and attention because—especially at this time—we are constantly overwhelmed by a continuous flood of information, so we need tools to filter the true from the false, the relevant from the irrelevant, the consistent from the inconsistent. 

Q: Can you share some information about your latest research projects?

A: One is the Inclusive Memory Project, which is a collaborative effort of seven academic departments of the University of Roma Tre. It combines innovative teaching methodologies, such as individual and collaborative writing assignments, with new digital tools to develop critical thinking, communication, and cooperation skills, particularly focusing on disadvantaged groups. [Ed. Note: More information about this project can be found at]

Another project is the development of a prototype to assess University Teachers’ Critical Thinking Skills. Pilot studies for this project are being done in both the USA and Italy). [Ed. Note: More information can be found at]

Antonella Poce is an Associate Professor (qualified as a Full Professor), in Experimental Pedagogy in the Department of Education at Roma Tre University in Rome. She is one of the keynote speakers at ICELW 2020 – International Conference on E-Learning in the Workplace, taking place at Columbia University in New York, June 10-12, 2020.

Evelyn Levine works as a Training and Staff Development Director for the U.S. Courts. She writes on worldwide learning and development trends in public and private sectors. She can be reached at