Preparing Tomorrow's Teachers to use Technology Grant 
(Funded by the U.S. Department of Education)

EXECUTIVE SUMMARY

In general, the results indicate that the first year activities were very successful. Of the 21 participants, only 3 (or 14%) expressed negative comments about the quality of the instruction and management of the project. These individuals perceived themselves as having more advanced technical skills and/or being more experienced with course alignment than their peers in the workshop. Each expressed somewhat different problems with the workshop (e.g., differing opinions on the quality work, perception that more time and effort were required than advertised, and issues of lack of clarity or efficacy). Nevertheless, these individuals acknowledged the value of the project and would like to see it continued in some fashion (e.g., new management, different and/or longer time, etc.). In fact, these individuals admitted that they learned some new skills and improved on others. It is noteworthy that the vast majority (approximately 95%) of participants reported significant improvements in various skills after participating in the workshop. This was most noticeable in the proportion of participants who reported significant changes in their multimedia skills. The most notable changes were their abilities to use "Blackboard", audio/video clips, Digital Drop Box, Power Point, Excel, Web technology, etc. Nevertheless, approximately 95% of the participants were able to make substantial improvement in aligning their courses.

All participants indicated that the project should be continued. However, in the event that the project is discontinued, a majority of the participants reported that they would continue to hone their skills even if it meant purchasing home equipment. Although the stipend was a very important recruitment incentive, approximately 95% of the participants were attracted by opportunities to enhance their multimedia skills and ability to align courses in view of State competency requirements and personal development.

Participants express concern about availability of and access to high quality technological equipment outside of the workshop facilities. This appears to be a University-wide issue. Participants claim few buildings have LCD projectors, laptop computers, or other basic equipment that would facilitate their ability to use what they learned in the workshop for the benefit of students. Furthermore, some participants encountered problems with the hardware, software or both while in the workshop or outside when attempting to complete assignments or practice. These conditions caused some participants to be concerned about student’s limited access to computers and other technology-related equipment because University labs are not always open 24 hours.

INTRODUCTION

According to the U.S. Department of Education (2001), the primary goal of the PT3 Project is "to improve the knowledge and ability of future teachers to use technology in improved teaching practices and student learning opportunities, and to improve the quality of teacher preparation programs." The overall goals of the Winston-Salem State University (WSSU) PT3 Project are consistent with those of the U. S. Department of Education. Specifically, WSSU saw its focus as building and sustaining significant improvements in its teacher education program. The Project leaders believed that faculty development was the single most critical factor in achieving that goal. As a result, the project was strategically designed to address faculty development needs.

The WSSU PT3 Project provided members of the faculty with an opportunity to participate in one or both workshops designed to meet the project's objective of improving teacher education. The two workshops were: "The Use of Multimedia Technology" and "Course Alignment." The leaders were particularly concerned about conducting a proper assessment of the various activities early in the project in order to make real-time corrections to better accomplish the end goal. The leaders saw evaluation as a cornerstone of the project. To this end, the project included a component that would address both formative and summative evaluation. In the spring of 2001, Dr. Willie Pearson, Jr. was contracted to conduct an external evaluation of the WSSU PT3 Project. This report represents a formative evaluation of activities implemented in the first year. The report is organized around three major sections: (1) methodology, (2) findings, and (3) implications of the findings.

METHODOLOGY

The Spring 2001, the external evaluator met with the Principal Investigator (PI), Dr. Edwin Bell and Project Director (PD), Dr. Madu Ireh to discuss the timelines, activities, goals, objectives and an evaluation plan for the first year of the project. It was agreed that the strongest evaluation plan includes both quantitative and qualitative data. As a result, data collection involved mailed surveys, site visits and personal interviews with all participants (including the PI and PD). Surveys for workshop participants and separate interview protocols for workshop participants, PI and PD were developed, pre-tested and revised before administering. A consent form was also developed and administered to each subject for signature before data collection. The principal evaluator conducted three site visits to the campus of WSSU. These visits involved observing the first orientation and a final wrap-up session for the workshops, and observing the laboratory facilities in the School of Education.

Data were collected from pre- and post-surveys assessing the workshop training, and from in-person interviews with workshop participants and one interview each with the PI and PD. Interviews were conducted between October 17 and November 20,2001 on the campus of WSSU (except one which was conducted at the administrative offices of the Forsyth County Board of Education). With permission, each interview was tape recorded (except where a respondent chose to speak off the record); Taped interviews were transcribed and sanitized of any information that would identify the interviewee.

Subjects. In year one, 21 individuals participated in the workshops. Of these, 48% are from the School of Education, while the remainder is from the School of Arts and Sciences. Sixty-seven percent are females who have been at WSSU approximately 8 years compared to 16 years for males. Women are concentrated at the adjunct, instructor and assistant professor ranks. Women are overrepresented among faculty involved in the Multimedia workshop and among those who enrolled in both the Alignment and Multimedia workshops. In terms of ethnicity, 38% are US -born Whites, 14% non-US born, and 48% are US-born minorities.

FINDINGS

The findings from the interviews and surveys may be categorized into the following major or recurring themes:  1) university infrastructure; 2) incentives and reasons for taking the workshops; 3) workshop organization; 4) student performance; and 5) networking. The discussion that follows draws heavily from the personal interviews.

University Infrastructure 

The hardware and equipment used received passing marks but certainly not a consistently strong endorsement.  In general, the University severely lacks adequate technology and equipment for faculty (not just the participants) to fully utilize the material and concepts addressed in the multimedia workshop. The underdeveloped University infrastructure is reflected, in part, by the fact participants reported that the campus only became wired within the last two years.  This lack is reflected in an interviewee’s remarks:  “The problem is that we lack adequate equipment...I don’t have a laptop computer in my class...If this does not change these workshops will be of little use in better preparing our students.” This is echoed in the comments of another respondent: “What I would really like is having a classroom already set up with the equipment needed to utilize the skills gained at the workshops...As is, we have to lug the machines to classrooms. This ends up taking 15 minutes, including set up time.”  One workshop participant succinctly described their experience:  “We had some problems with some of the work stations…they didn’t work.”  Apparently, some problems occurred with both laptops and desktops. Equally important as the lack of adequate technology and equipment is the inadequate access to the technology and equipment that the University does have. One respondent reported: “Although we have a lab open around the clock, access for the students is not what it should be.” 

The absence of classroom facilities and accessibility to equipment suggests that the lacks the infrastructure to fully support the goals of the PT3 Project. In fact, other institutions with the University of North Carolina system have facilities that are far more advanced than those of Winston-Salem State University. Indeed, the University lacks a comprehensive technology strategy—and it shows, according to the workshop participants.

Incentives and Reasons for Taking the Workshop 

Although the stipend for taking the workshop was $3000, it would be a mistake to assume that this was the only incentive to participate in the workshop. According to one respondent:  “I was planning to revise the (course) outline and this seemed to provide an opportunity to obtain more information and do a better job...and the $3000 is certainly an incentive.”  Another respondent said “I didn’t get any sense that there were people there just for the money.”  However, one respondent noted the sharp reduction in the stipend for the second phase. Apparently, a number of respondents had not been paid at the time of the interview.  These respondents attribute this to a consistent pattern of inefficiency in the accounting office.

A substantial majority of respondents mentioned knowledge and skill development as compelling reasons to take the workshop.  One respondent’s reason was related to an interest in technology:  “I am interested in anything to do with technology.”  Another’s reason was the importance of technology for students:  “There are numerous demands on our graduates to be technologically efficient and proficient…and this is an opportunity for our departments to make our graduates competitive.”

The workshops provided an opportunity for participants to gain additional knowledge and technical skills. According to one respondent, “It is probably the best workshop of its kind that I have participated in...more because of substance than form.”  Some workshop participants cited their improvement in course consistency: “My courses are now more internally consistent...the objectives match the activities.”  Others mentioned the benefit of consolidation: “...it helped me develop the modules that are now being team taught in my department...we were able to condense 27 semester hours into three nine-hour modules.”  One participant benefited insofar as the workshop “…has expanded the way that I reach students because it strengthen some old skills and introduced me to new concepts.”  For others, participating in the workshop opened up a world of possibilities that they did not think existed.  One participant said that:  ”What I had been doing was so rudimentary compared to what was required to be done...” This also speaks to the high performance set by the PI and PD. Another remarked that: “I have seen people doing things that they were not doing before.”

The workshop enhanced some participants’ ability to more clearly articulate course requirements and relate them to performance objectives.  One respondent noted that “Prior to the workshop, I could not write instructional or performance objectives according to Bloom’s Taxonomy.” Another found that “I can develop a rubric which clearly spells out expectations for a culminating project, the different levels of performance, and the criteria assessing the product at each level.” Still another found that

“I had to think about everything that I am asking students to do.  If it doesn’t relate to standards, then I am leaving them out.  I feel better about requiring them to do things with technology because I feel better about my ability to do it myself and show them how.”

The workshop benefited even those who already had experience using technology:  “Although I had a background in the use of technology, I still learned a lot.”  Even a critic said, “I want to give them credit because I am more aware of the possibilities of inserting more things into a multimedia presentation.”  Indeed the introductory exposure to multimedia technology provided by the workshops were the catalyst for some participants to get more:  “Once I learned how use the new multimedia technology, I wanted to go to more advanced technologies like animation.”

Workshop Organization

All workshop participants clearly understood and articulated the general goals of the project as “helping professors integrate technology into their courses to better help students use technology in appropriate ways when they teach and focus the curriculum on the standards for the professional organizations and the state.”  Participants identified several organization issues:  timing; format/structure; and coordination.

Timing.  There were other technology-oriented workshops being offered on campus at the same time as the PT3 workshop.  Some participants commented that they were distracted by other obligations because the workshop began while the participants were still in school. However, one respondent said that “having one of the sessions on the weekend really worked well.”

Format/Structure.  A substantial majority of participants reported that the structure of the workshops was excellent because presented the material in clusters.  Some of it was overview.” One participant noted that “The workshops helped focus on what we wanted students to learn.”  Most respondents commented that workshops were well organized.  The hands-on structure was important for all participants.  The comments of one respondent are instructive:  “it is one of the best technology workshops I have taken because it is hands-on.”  Although a few participants believed that the workshops should have been structured to accommodate the different levels of expertise among participants, a vast majority of participants believed that including participants with varying skill levels facilitated learning, especially among those with limited exposure and experience with the workshop topics. Still other participants said that they would have benefited from more time for hands-on instruction or practice because they needed the repetitiveness until they felt more confident.

Coordination.  Respondents made very positive comments about the PI and PD.  One respondent’s comments characterize the sentiment of the majority: “They could not have found a better coordinator than Madu.  He knows how to work with the faculty. He is very patient.”  Another respondent said that the Bell, the PI, “gives Madu the power to follow the program as outlined.  He doesn’t try to act like he is in charge.  Both of them listen.”

Instruction. Some workshop participants praised the instructors for the way in which they worked closely (one-on-one) with the participants.  One participant said:  “The instructor would speak in general, then show me exactly what to do and how to do it.  That was very useful.”  Another recounted this experience: “Madu spent 20-30 minutes taking me step by step.” Still another said: “The instructors responded to anything that I wanted to know.” A workshop participant said: “I got so much out of the workshop...I am now a better teacher because I look at teaching from a different perspective.”

While virtually all workshop participants expressed such praise, one participant countered: “I would not have ever entered the project had I realized that it would be so dictatorial and closed-minded.” Another participant expressed “…concerns about the leadership and how certain things were handled...For example, the inflexibility to meet the needs of the other disciplines...it was like my way or the highway.” In a backhanded compliment, one participant said: “__ is an excellent teacher but real nit-picky but If __ had not been nit-picky my document would be as good as it is!” Some participants complained about—but ultimately appreciated—the amount of work required of them:  “I think that if I had understood at the very beginning all of the things that were required, I might not have signed on...It is a lot work...it is so involved.... but I learned a lot and I am proud of the document that I ended up with.”  The PI agreed that the performance standards were set very high.  This was the usual expectation of faculty.  Nevertheless, the above-mentioned comments strongly suggest that most participants rose to the occasion.  Indeed, a majority of participants spoke glowingly of their surprising achievements.  They were proud they were able achieve well beyond their self-imposed expectations.

Student Performance

Respondents discussed the impact of their workshop participation on their students.  One reported; “I have already seen dividends with my students.” One workshop participant said:  “My students are saying that just by being in the online course, their technology skills are being developed.” Another workshop participant reported now being able “to give students better assistance because I also require them to use technology and to demonstrate it.  They have to do their group project using PowerPoint in class.”  Moreover, a respondent noted that the principals have noticed the differences that workshop has made: “The principals are telling me how different the lateral entry teachers are now that they have been exposed to multimedia and aligned courses.”  Finally, one respondent pointed the importance of using the appropriate technology in order to take into account varying learning styles:  “I had a student who is very bright, but is learning disabled...so, the on-line course was difficulty for him...I learned that the technology must take into consideration various learning styles.”

Network

A major benefit from participating in the workshop was the opportunity to become part of new networks:  “I am now part of network of people who are interested in multimedia technologies. Had I not been part of the PT3 workshop, I probably wouldn’t be a part of that (network).”  Another major benefit was forming interdisciplinary collaborative groups: “Before the workshop, we really didn’t have much contact with the people in Education. Now, we (math/science faculty) are planning to form a program so that we can prepare science teachers.”

Some commented on how the structure of the workshop was the catalyst for building networks:  “I like to work with people from a wide range of backgrounds.  I thought this was a strength because it helped team building because we had different strengths.”

RECOMMENDATIONS

Train-the-trainer model.  The participants’ comments suggest one model for this kind of workshop:  a “train-the-trainer” model.  Several workshop participants eagerly shared their enthusiasm for the technology and their newly acquired skills with other faculty members who could not attend the workshop.  This created a “ripple effect” or “value-added” insofar as more people benefited from the training than those who attended the workshop. 

Vary workshop structure.  In the future, the PD should consider varying the structure of the workshop so that some demonstrations will be presented to all (mixed ability groups), and different topics will be presented to groups according to the skill level of their members. In addition, the PD should add in more practice time—and make certain there is enough equipment available for practice.  The PD should consider follow-up or booster training as a way keeping the skills levels up until more participants have assess to adequate equipment.

Until the University addresses this issue, any progress towards the goals of PT3 will be marginal at best. Moreover, the goals of PT3—i.e., use of technology and implementation of course standards—should exist across all disciplines.

Participants recommended that future workshops should include examples of poorly and exemplary aligned courses. Addition, many participants reported that they could have benefited from a discussion of the number drafts or time investment to move from a poorly aligned to an exemplary course design. Most participants believed that this would provide an early context of what was expected performance.

Learning Styles.  The PD may wish to address the issue of using the technology appropriately to enhance the learning environments of individuals with different learning styles.  In particular, more attention should paid to students with disabilities.

Institutional Technology Plan.  To the extent possible, the PT3 Project should be integrated into the overall strategic technology plan of the University.  This may provide an opportunity to better coordinate workshops.

Community Building.   The PI and PD should consider establishing some form of an electronic community.   Regardless of continuation of the PT3 Project, all participants (including critics) indicated a desire to continue aligning courses and developing their skills in multimedia instructional technology.  This further demonstrates the effectiveness of PT3 project in first year.