|Peculiarities||SocioSite||Subject Areas||Society||Search||About us||Contact|
In the previous chapter we have seen that the structuring of the learning experiences (i.e. telecoaching) is a complex process that comprises almost all levels and aspects of the educational practice. In order to evaluate this telecoaching process in a systematic manner I have chosen for the transformational perspective on education. In this chapter this analytical perspective will be used to specify (a) a general evaluation approach for telelearning processes, and particularly (b) a method for evaluating the telecoaching which has been practised in the TAET-course. I will start with the construction of a more general analytical framework that can be used to evaluate the entire process of telelearning.
In educational research of online courses we can differentiate seven analytical levels which have to be taken into account. Each level has its own peculiarities, potentials and problems. In the absence of a funny acronym, I will call it the SLNM: Seven_Levels_ No_More. In the next table these seven levels of telelearning are presented in a scheme in which the main features of the levels and the corresponding potential problems are outlined.
Ambiguous, insufficiently demarcated assignments Structured & non-structured assignments No access to computer or access to 'underachieving' computer
Intricate system of evaluation instruments and procedures
No/inadequate evaluation tools; unsystematic use of evaluation tools
Intensity and temporality of individual and collective return-information
Lack of or inadequate feedback: too little, too late
Lively learning community that motivates individuals and stimulates them to cooperate in collaborative learning
Failing community building: boring, irregular contact, etc.
Structuring and content of assignments
Badly composed series of assignments
Quality & transparency of course information
Poorly specified course content; unclear level of expectations, etc.
Basic computer, network & internet abilities of students
No or not enough computer, network and/or web literacy
Access to network and to personal computer
No access, slow access, disturbed access to network
Ambiguous, insufficiently demarcated assignments
Structured & non-structured assignments
No access to computer or access to 'underachieving' computer
This SLNM-model can be used for (a) the construction of a theoretical framework and literature study, (b) the development of a questionnaire for TAET students and teachers, and (c) the empirical analysis of TAET course. In this thesis I concentrate primarily on the community and the coaching level. Since coaching is also inextricably bound up with course content and assignments, these levels will also be discussed. Other levels will only be touched on briefly, either due to time constraints or due to a lack of direct connection with the subject of this thesis: What does coaching 'look like' in a virtual learning environment and how can coaches stimulate the development of a vital learning community.
My hypothesis will be that the effectiveness of telecoaching depends primarily - though not exclusively - on the extent in which coaches succeed in moderating collaborative learning processes between and among students (and instructors). I presuppose that the creation of a learning community during a course is one of the most critical success factors of tele-education.
My empirical investigation will be directed by this main question:
what activities and roles of coaches have an impact on the learning process
as well as on the development of a learning community?
This question can be subdivided into these subquestions:
Below I will indicate what aspects of the different levels I believe are important and can be analysed and what I will leave out due to time constraints (pragmatic reason). For some levels I have worked out some indicators to be measured. Some of the questions that were asked in the questionnaire are inserted behind these indicators.
The infrastructural level is defined by the conditions that facilitate good (fast, reliable etc.) access to personal computer, local networks, and the internet. The infrastructural level doesn't really concern the main theme of my thesis. Yet, some TAET students didn't possess a PC when they started the course and were enabled by the faculty to borrow one. And as we will see in the next chapter, some students had problems getting a reliable connection with the TAET-site.
In the questions on infrastructural level respondents were asked to briefly state the problems they had with the technology (network, computer) and also indicate positive experiences. This was specified in the following questions:
Basic Skill Level
The basic skill level is defined by the specific state of the basic computer, network, and internet abilities of students. In the studies I've read so far a recurring problem is pointed out: the lack of basic computer skills and webwisdom, combined with unfamiliarity with the electronic learning environment. Some studies report students' frustrations due to technological problems and suggest an introductory course before they really get started. In such introductory courses students can acquire basic computer skills, learn how to surf on and search the web, and get familiar with the electronic learning environment. We know in advance that quite a few TAET students had problems originating in a lack of computer- or webwisdom, especially with building a website in one of the first courses. They also had to become familiar with the electronic learning environment.
The questions on basic skills can be specified as follows:
Course Content Level
The course content level is defined by the quality and transparency of the material that is used in an online course. Course content relates to:
On all these four items students will be asked some questions.
The assignment level is defined by the structuring and content of assignments. The assignment level can be questioned and analysed as to:
It will not be easy to answer questions for this level for TAET students, because there were generally many assignments per subject and memory may have faded.
The community level is defined by the imagined and experienced community that motivates students and stimulates them to engage in collaborative learning. The danger inherent in distance learning is that the social aspects of learning are getting lost. Social embedding of a learning process is of paramount importance, not only for an efficient learning process, but also for future professional practices in which collaboration is of vital importance. In some studies students' wishes to be part of a learning community is explicitly reported (transformation from outsider to insider) .
Cooperation can be measured for example as to
Sources of information for TAET-students are
The coaching level is defined by the intensity and temporality of individual and collective return-information. Coaching refers to coaching activities of the instructor. In general we can say that coaching has two basic components: (a) instruction and (b) process control. These components will be subdivided later (e.g. different roles of the teacher: instructor, judge, motivator, provider of examples, advanced organiser, etc.). The instructor is the person to develop and control the quality of the (collaborative) learning environment and learning process and the one who can moderate the social embedding of the learning process.
In the teachers' questionnaire questions concerning available time for coaching and views on coaching will be included (for example: "what has been - or should be - your role in stimulating a collaborative learning process?") . In the students' questionnaire explicit questions about expectations and appreciation will be included.
The evaluation level is defined by the intricate system of evaluation instruments and procedures. In my empirical investigation I will not enter into the discussion about course evaluation. My investigation is itself a form of evaluation research.
Yet I will complete my TAET research with conversations with some experts of the Open University [Heerlen] on the issue of coaching strategies and practices. My hypothesis will be that the introduction of tele-education generates an increasing coaching need of students/trainees. A kind of 'revolution of rising coaching expectations'. Telelearners who present a paper or assignment expect an adequate, personal and above all fast reaction. My question to the experts will be how they practically handle this increasing demand of adequate, personal and fast feedback. (a) What problems do they identify: where do things go wrong and why? (b) What are the strategies to solve these problems: how can you realise a good coaching that comes up to the coaching expectations of students in a situation of scarce means?
In my qualitative research project I have drawn upon the following sources of insight and information:
The urls of the questionnaires were sent to my fellow students and teachers in a letter sent by email, explaining the purpose of my research. In the weeks after I was so fortunate to receive quite a number of replies.
Of the 16 students 11 responded. 3 out of 16 students dropped out and although I asked them to respond I received no reply. I did not fill out the questionnaire myself. In the period when the questionnaires came in it turned out that 2 had disappeared somewhere in cyberspace. I had not been clear in my instructions that the questionnaire had to be filled out online. This omission may be due to the fact that I have a fast cable connection and can remain online all the time. Quite a few of my fellow students have a telephone connection and limit their time online for economical reasons. That might have been the problem. One student tried to file it halfway the process of answering the questions. When she tried to retrieve the answers it appeared that they had got lost. I approached the two students whose questionnaire had disappeared and came to an agreement with them. I had an interview with one of them at her home and a telephone interview with the other (who had sent in the questionnaire twice already!). I sent a reminder twice to the students who hadn't responded (4, including the 'dropouts'). I asked them kindly to give me a reason for not filling out the questionnaire if they couldn't or wouldn't respond. From none of them I received an answer.
Of the 19 teachers who taught in the TAET course 16 responded. I sent them a reminder once and also asked them to give me a reason for not filling out the questionnaire if they couldn't or wouldn't respond. The same story counts here: no replies from 2 teachers. I assume no teachers' questionnaires got lost in cyberspace, at least not to my knowledge.
The main problem with the questionnaire was that the senders received no 'copy' of it. This was not possible because the senders were outside of the domain of the server of the University of Amsterdam, where the questionnaires were sent to after having been answered. This is a problem to be avoided in future research.
|Peculiarities||SocioSite||Subject Areas||Society||Search||About us||Contact|