©Heidi Bratton Photography
Of the many volunteer jobs I’ve taken to promote the Church, one that I found most interesting was chauffeuring a well-known Catholic convert and evangelist from appointment to appointment. One day, I drove him to a television station where he was interviewed by his bishop about his conversion to Catholicism. On air, the bishop said he assumed the evangelist came into the Catholic Church because of the beautiful and holy example of the many Catholic laity the evangelist had met over the course of his life. Diplomatically, my evangelist friend said nothing, because nothing could have been further from the truth.
In my ten years as a Catholic, I have met a number of priests, bishops, writers, and other Catholic leaders who agree that the knowledge of the typical Catholic layperson about his or her faith is disgraceful. Like my evangelist friend, many of us converts to Catholicism came into the Church in spite of the poor formation of the Catholics we knew, not because of them. We came into the Church because of the consistency of Catholic teaching with the Bible and history.
There are many historical reasons why American Catholics are so poorly catechized, and I’m sure someone has written the book. There is, however, one thing RCIA and CCD instructors can do today that, if they are not doing it now, will dramatically improve their efforts at catechetical instruction overnight. While this essay does not allow enough space to explain this technique in detail, it will provide an introduction and a free resource to get you, as instructors and coordinators, started.
There Will Be A Test
Fr. James Cronk would occasionally scan the class, look us in the eyes and gleefully announce: “Just remember, there will be a test.” As a candidate for entering the Church, I was attending my first RCIA[1] program. It took a while before we understood Fr. Jim’s meaning, its truth, and how it applied not only to each of us in the class, but to him, and all RCIA coordinators, instructors, facilitators, priests, and bishops. Yes, there will be a test — and, I might add, it will be scored.
In the adult corporate training and communication industry — where I spent the better part of three decades as a producer and creative director — my group’s paychecks were tied to how efficiently, effectively, and productively the students under our care were educated. Our results were never based on our senses or feelings, or someone’s subjective opinion. Our clients, who collectively were spending millions of dollars, wanted more assurance than my cocky, nodding head.
So, we evaluated our training in several different ways and, based on the results of those tests, we modified our methods and techniques. In a temporal sense, we were very successful and our business lucrative as we worked with a breadth of demanding customers from the NASA astronaut corps to Harley-Davidson dealers to the heads of the major automotive companies and their divisions. The point of our evaluation effort was to ensure the return on investment (ROI) that our customers were making.
Lest you think that such ROI accountability doesn’t fit the world of Christian discipleship, I will remind you of Christ’s parable of the talents, and what happened to those who did not return double the investment of their master. There was a test… and, those who failed it “…were cast into utter darkness” (Matt. 25).
Four Levels, Two Groups
In the instructional design industry there are four levels of instruction, evaluation or testing referred to simply as Levels 1, 2, 3, and 4. At each level there are two different groups being evaluated: (1) the instructional designers and instructors; and (2) the participants.
Levels 1 and 2 are conducted at the time of the instruction’s delivery — either at the end of a single class session, or, in the case of longer instructional periods, occasionally throughout. Level 1 evaluates the instruction from the student’s perspective and Level 2 evaluates the student’s comprehension from the instruction’s perspective. (Notice that the instruction has two parts: (a) the instructional design that includes the structure of the content, presentation method, and overall methodology, and (b) the instructor.
Levels 3 and 4 are conducted later. Level 3 testing is typically administered a month after the class ends, and, like Level 2, evaluates the student’s knowledge or comprehension. Level 4 is the most difficult and expensive to administer because it attempts to evaluate, a month to a year out, the student’s change in behavior as it relates to the instruction. That is nearly impossible because of the many variables in the student’s life that can affect behavioral change.
Ultimately, there are two goals in these evaluations. The first and primary purpose of the evaluations, is not to grade the student’s performance, but to improve the instruction. Yes, that’s right. Unlike the grades you received in elementary school, high school and college, these evaluations place the learning onus on the instructional designers and instructors. The second purpose, of course, is to test the student’s progress at learning.
The Problem from My Seat
I converted to Catholicism from Evangelical Christianity in 1997-1998. Because of my professional interest in learning and education and because of various Church requirements and suggestions, I’ve been involved in several RCIA, CCD, and other Catholic educational efforts. Distressingly, my direct observations in too many of these programs have revealed the absence of the most basic communication protocol.
I have been in Catholic education classes where you could not understand a word of what the instructor was saying, or the level of the reading material was inappropriate for the class, or the instructional methodology used for a class of 7th graders was geared to that of an alert, motivated adult on Double-Jolt Cola.
I have also attended far too many sessions where the instructor had a degree in theology but had few facilitation skills and put the class to sleep. In all of these cases, regardless of the sincerity of the instructor or the accuracy of the theology presented, the instruction might as well have not occurred.
If we’re concerned about passing on the faith to the next generation — partly at least through classroom instruction — then it is imperative that we teach in a way that learners can and will learn. Just “showing up,” sincerity, and prayer are no substitutes for basic instructional and communication skills.
What Should Be Done NOW
As we approach the end of another year of RCIA instruction, now would be a good time to administer an important piece of this evaluation strategy — the Level 1 evaluation. At the same time, a Level 2 test can also be administered. Although the Level 2 evaluation is often seen as intimidating or offensive by students, and while many RCIA instructors may feel that such testing demeans the students and treats them like middle school children, the fact remains that testing helps the students understand what’s important and helps them learn.
For instance, if before the instruction you were to administer a pre-instruction test of the critical learning objectives, regardless of how well or poorly the students do on the pre-test, they would know what was important to remember when the instruction occurred. This kind of “pre-testing” serves an important instructional objective — it helps the student establish the “hooks” upon which to “hang” the information when it comes.
Then, if the same Level 2 test is administered after the instruction, several wonderful things occur. First, the instructional designers and instructor can compare the pre- and post-tests to see if learning, at least in the short term, occurred. Second, if there is not a dramatic improvement in the scores, then careful investigation into the instruction needs to be made; and that is when the Level 1 evaluation is most helpful. By itself, though, the Level 1 evaluation provides great benefit to class coordinators, developers, and the instructor.
The Level 1 Evaluation
A Level 1 Class Session Evaluation that can be used for RCIA instruction can be found at the bottom of Nineveh’s Crossing’s Home page [http://www.NinevehsCrossing.com]. Look for the link to “Best Practices for Faith Formation.”
The Level 1 Evaluation found there can be used after each class session or whenever the content, instructor, or venue changes. Level 1 is often called a “smile” sheet because its primary purpose is to discover how satisfied the students are with the instruction as it was presented. In order to get high grades on a Level 2 evaluation (student head-knowledge of the content), a high Level 1 is absolutely necessary. If the Level 1 evaluation scores low, it is not possible for Level 2 to be high, unless the student learned the material somewhere else. If students are to learn in a classroom situation, there first has to be good communication, and the Level 1 evaluation can help religious education directors achieve that.
The Level 1 Evaluation, referenced above is a short questionnaire with 12 questions that provide both quantitative and qualitative answers. After asking for basic information about the date, class, instructor and topic, and reminding the student that there are no wrong answers on this “test”, the following questions are presented. The first eight ask students to Disagree or Agree on a 1-5 scale with statements about their learning experience, with opportunity of explaining their ratings on the back of the form. The statements include:
1. This session was of interest to me and met a need in my life.
2. I could see and hear the instructor and visual aids well.
3. The lesson was presented at a level that I understood.
4. The instructor was well prepared to teach this lesson.
5. The instructor’s appearance and attitude were engaging.
6. The classroom was comfortable and conducive to learning.
7. The reading material that supports this session was helpful.
8. I would encourage others to take this session.
Questions 9 through 12 ask for open feedback about what in the class needs to be deleted, added, used less, or used more, to help improve the class session.
Administration of this Level 1 Evaluation at every session, and then paying attention to and possibly implementing the suggestions you get, will dramatically improve the effectiveness of your instruction to the next generation of Catholics.
A free PDF Download of the above evaluation with completion instructions — ready to hand out — is available at http://www.NinevehsCrossing.com. Scroll to the bottom and click on the BEST PRACTICES link.
By the way, there is a Level 5 Evaluation for Christians. It’s the “test” Fr. Cronk was reminding us about. It takes place at the “Pearly Gates,” and it will be scored.
[1] RCIA stands for Rite of Christian Initiation for Adults, a series of parish classes held from September to Easter that intend to teach adults all things Christian and Catholic in order to prepare them to enter the Church at the Easter Vigil through the reception of three sacraments: baptism, confirmation, and communion. RCIA participants are generally composedof three types: catechumens (the unbaptized), candidates (baptized but unconfirmed), and sponsors (Catholics who assist the catechumens and candidates through the six month process).
(© 2011 Stanley D. Williams, Ph.D)
