European Journal of Open Education and E-learning Studies ISSN: 2501 - 9120 ISSN-L: 2501 - 9120 Available on-line at: www.oapub.org/edu 10.5281/zenodo.164131 Volume 1│Issue 1│2016 ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMSi Betül Tonbuloğlu1ii, Aysun Gürol2 PhD Student, Yildiz Technical University, College of Education, 1 Curriculum and Instruction Department, Istanbul, Turkey Yildiz Technical University, College of Education, 2 Curriculum and Instruction Department, Istanbul, Turkey Abstract: Evaluation of opinions and satisfaction levels of students who continue their education through distance education programs is crucial in detecting the problems encountered in the programs and in improving them. The purpose of this study is to identify student evaluations and satisfaction levels concerning the pedagogical dimension of distance education programs. The single screening model was used in this quantitative study. The criterion sampling method, one of the purposeful sampling methods, was used in determining the universities whose distance education system would be examined. The study population consisted of universities offering distance education in Istanbul; the study sample consisted of 5 universities, among universities with at least 3 years distance education experience, which offered at least one of the associate degree, bachelor’s degree or master’s degree programs online and which had various learning management systems. The satisfaction questionnaire, developed by the researcher, was used as the data collection instrument; the pilot study of the questionnaire was applied on 194 distance education program students. The Cronbach alpha coefficient was observed to be 0.988 and the questionnaire was accepted to be reliable. Data were collected from 533 students studying in the distance education programs in the universities within the study sample. Data collected from the questionnaire were analyzed through the SPSS (Version 20) software. Findings suggest that students find it necessary to develop distance education programs in course design and content, the teaching and learning process, educational materials, the support provided and This article study was derived from the doctoral dissertation of ”et(l Tonbuloğlu and was supported based on the number 2015-09-01-DOP01 project recognized by the BAP Commission. i Copyright © The Author(s). All Rights Reserved Published by Open Access Publishing Group ©2015. 152 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS assessment-evaluation. It is suggested that evaluating distance education programs in regular intervals and making improvements based on evaluation findings will enhance the quality of the education provided. Keywords: distance education, program evaluation, satisfaction questionnaire, student opinions Introduction Distance education is defined as an educational practice carried out between a teacher and student in a different time and place setting and which is based on student efficiency and self-learning principle Karataş, . Middlehurst and Woodfield identify the common characteristics of distance learning as; a separation between teacher and learner (in time, place or both), accreditation or certification by a learning institution or external body, Use of varied media for course delivery, communication mechanisms between teacher-learners and learners-learners, optional personal meetings and use of industrialized processes. The term distance education consists of distance teaching and distance learning and integrates the two terms. Distance education explains the course development process of a distance institution which prepares educational materials for students. Distance education was evaluated according to student views within this process. Because learning is a product expected from the learning-teaching process, distance education should be considered within the student’s perspective on the process Uşun, . Lentell (2012) states that features that distance education functions should carry as; well-structuring assistance systems such as developing learning resources and systems convenient for learner objectives, making flexible for administrative arrangements those who work part-time and who are distant, selecting appropriate technologies for students receiving distance education to communicate with their teachers and giving feedback sufficient feedback to students on time. Distance education program is defined as the activity conducted by educational institutions through a syllabus prepared with a plan to help students complete their education on their own Uşun, . Program evolution studies are necessary for maintaining, completing or revising the programs (Worthen, Borg and White, 1993), for determining and developing the weaknesses and strengths of the program and for determining the problems, educational needs and desired educational outcomes of the program (Sanders and Nafziger, 1976). While face-to-face and distance education program evaluations are similar, there are also primary differences due to the structure European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 153 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS of distance education. These are openness to evaluation, things to be evaluated, success criteria and evaluator (Kaya, 2002). Cost can be added to these factors as well. Rumble (2001) lists the factors affecting cost as the number of students, the number of programs provided through distance education, duration of programs, use of materials with copyright, the number of personnel, technologies being used and material development Evaluation of opinions and satisfaction levels of students who continue their education through distance education programs is crucial in detecting the strengths and weaknesses of distance education programs and the problems encountered in the programs and in improving them. Rivera, McAlister and Rice (2002) emphasized that student graduation is crucial for the long term success of web-based courses and stated that it should be researched. The purpose of this study is to identify student evaluations and satisfaction levels concerning the pedagogical dimension of distance education programs. With this respect, answers for the following questions will be sought: 1. What level are student evaluations on the overall introduction of the courses? 2. What level are student evaluations on course design and content of the courses? 3. What level are student evaluations on the teaching and learning process? 4. What level are student evaluations on the technology being used? 5. What level are student evaluations on instructional materials? 6. What level are student evaluations on the support provided? 7. What level are student evaluations on the assessment-evaluation dimension? Method Study Design The single screening model was used in this quantitative study. The single screening model involves describing current states related to the research subject one-by-one and determining the behaviors, attitude, expectation, need and level of knowledge related to the research subject (Karasar, 1999). Study Sample The study population consists of universities in Istanbul which offer distance education. The criterion sampling method, one of the purposeful sampling methods, was used in determining the universities whose distance education system would be examined. The logic of the criterion sampling method is to examine and review every condition, which is a common strategy in quality assurance practices, that makes up for various predetermined significance criteria (Patton, 2014). The study population consisted of universities offering distance education; the study sample consisted of 5 universities, European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 154 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS among universities with at least 3 years distance education experience (who started giving distance education back in year 2012), which offered at least one of the associate degree, bachelor’s degree or master’s degree programs online and which had various learning management systems Data were collected from 533 students studying in the distance education programs in the universities within the study sample. Demographic information on the study sample is given on Table 1. Table 1: Demographic information of the study sample Demographic Information Features Number Gender Female Male 362 171 67.92 32.08 Age 19-25 26-35 36-45 46-65 322 138 59 14 60.41 25.89 11.06 2.62 University State Uni. Private Uni. 261 272 48.96 51.03 Educational Level Associate Deg. Bachelor’s Deg. Master’s Deg. 175 277 81 32.83 51.96 15.19 Name of Program Child Development Business Sociology Public Relations Law Secretary and Office Management Economics Geography Social Services Hospital and Health Facilities Management Marketing Others (Various 37 departments) 85 68 39 39 28 15.94 12.75 7.31 7.31 5.25 28 17 14 12 5.25 3.18 2.62 2.25 10 193 1.87 36.21 Grade Preparatory 1 2 3 4 Dissertation Stage 2 266 108 67 70 20 0.37 49.9 20.26 12.57 13.13 3.75 Working Status I am Unemployed I am Part-Time Employed I am Full-Time Employed 237 69 44.46 12.94 227 42.58 European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 % 155 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS 68% (362) of the students were female and 32% (171) were male. 322 people were aged between 19-25, 138 people between 26-35, 59 people between 36-45 and 14 people between 46-65. The number of students studying in the departments of Child Development, Business, Sociology and Public Relations were higher and year 1. and 2. students constituted 70% of the sample. The number of students working in an ordinary job was higher than the number of unemployed students. Data Collection Instrument In this study, a satisfaction questionnaire was developed by the researcher through the distance education accreditation standards set by 16 national and international accreditation institutions, the students receiving distance education in the universities of the sample were accessed through this questionnaire and their opinions and satisfaction levels on distance education were examined. Thomas (1998) defines questionnaire, which is one method of evaluating student satisfaction, as a data collection instrument consisting of questions which identify the life standards, beliefs, behaviors and attitudes of people. Büyüköztürk (2005) states that questionnaires are commonly used in social sciences in collecting data and are defined as a list of questions; and underlines that this data collection technique offers many advantages when compared with techniques such as observation and interview. These advantages can be listed as being applied on larger groups in different regions very rapidly and being low-cost. The draft form of the 57-item student satisfaction questionnaire was discussed by 3 instructors, one in the department of Computer and Instructional Technologies Education, one in the department of Distance Education (coordinator) and one in the department of Educational Programs and Teaching; the question items were arranged based on expert opinions of the instructors and the number of questions was reduced to 36. The reviewed form was e-mailed again to the instructors and was confirmed. The pilot study of the questionnaire was applied on 194 distance education students, reliability of the data was examined, the Cronbach alpha coefficient was observed to be 0.988 and the questionnaire was accepted to be reliable. The first section of the questionnaire consisted of 10 questions related to the participants’ demographic features, the second section consisted of questions related to student satisfaction levels concerning distance education, the third section consisted of one open-ended question related to student suggestions on distance education (and one question in the pilot study directed to student opinions on how they find the questionnaire). In the five point Likert type scale, the participants were asked to mark European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 156 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS the options stated as I Strongly “gree, I “gree, I Partly “gree, I Disagree, I Strongly Disagree . Data Collection and Analysis The satisfaction questionnaire, which was directed to the students receiving education through distance education in the universities within the sample, was sent to the students via e-mail and delivery during the Spring and Fall terms of 2014-2015 academic year. The coordinators and instructors working in the distance education units of the universities within the sample were asked to convey the questionnaire to the students through the distance education system or e-mail. The questionnaire was also distributed to the students in the exam places on dates when the students had distance education exams, questionnaire findings were gathered from 533 students through e-mail and distribution. Data collected from the questionnaire were analyzed through the SPSS (Version 20) software. Cohan, Manion and Morrison (1997) state that after collecting the data whether the questions are fully answered, whether the answers are appropriate for the questions and whether or not the coding process for the answers are the same for each form should be checked. Questionnaire data underwent necessary controls, each questionnaire item was evaluated separately and the percentage, frequency and arithmetic mean of each item were calculated through the SPSS 20 software. Openended questions were analyzed through the content analysis method. Answers for each question were interpreted according to the results. The arithmetic means were interpreted through 0.80 (5/4) intervals (1.00-1.80 very low, 1.81-2.60 low, 2.61-3.40 medium, 3.41-4.20 high and 4.21-5.00 very high). Intervals of the results were checked and student evaluations on distance education programs were explained. Findings 1. Findings concerning student evaluations on the overall introduction of the courses When the satisfaction questionnaire conducted on the students is considered, it is evident that student opinions on the overall introduction of distance education courses are at I Partly “gree- option. Findings on student evaluations are given on Table 2. European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 157 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS I Strongly Agree (5) 3,15 73 13,7 53 64 12 67 174 32,6 157 153 28,7 180 69 12,9 76 3,29 % 9,9 12,5 29,4 33,7 14,2 3,04 f % 67 12,5 84 15,7 192 36 136 25,5 54 10,1 I Disagree (2) f % f I Strongly Disagree (1) I Agree (4) 1. The syllabus which offers a clear description of the course can be accessed. 2. We are informed on issues related to the courses such as the instructional program, prerequisites of the courses, competences required for the course, how courses will begin and where all the course elements are. 7. The directives for how to attain the learning objective are sufficient and easily comprehended. Average (0-5) Questionnaire item I Partly Agree (3) Table 2: Findings on student evaluations on the overall introduction of the courses It is evident on Table 2 that the majority of the students are irresolute about the comprehend ability and sufficiency of the directives on accessing the instructional program which introduces the courses, the information given about the course subjects and the directives on how to attain learning objectives. 2. Findings concerning student evaluations on course design and contents It was observed that the average of the answers given to three of the seven questionnaire questions for this research question was between 1.81-2.60, in other words at low level; and the average of the answers given to the remaining four items was between 2.61-3.40, in other words at moderate level. Findings on Student evaluations are given on Table 3: European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 158 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS Table 3: Findings on student evaluations concerning course designs and contents of 2,52 149 131 154 68 31 % 27,9 24,5 28,8 12,7 5,8 2,54 f % f % f % f % f % f % 143 26,8 146 27,3 71 13,3 63 11,8 60 11,2 63 11,8 137 25,7 149 27,9 88 16,5 70 13,1 71 13,3 86 16,1 135 25,3 130 24,3 184 34,5 156 29,2 162 30,3 156 29,2 71 13,3 69 12,9 138 25,8 169 31,7 173 32,4 158 29,6 47 8,8 39 7,3 52 9,7 75 14 67 12,5 70 13,1 2,40 3,02 20. Course contents are up-to-date. 3,23 22. The language used in courses is comprehensible. 3,21 26. It is easy to explore the online course setting. 3,16 I Strongly Agree (5) f I Agree (4) I Partly Agree (3) 9. The courses we receive in distance education fulfill our needs. 10. Study groups and other activities that promote collaboration are used in course designing. 19. Course contents are arranged from concrete to abstract and from easy to complex. I Disagree (2) 8. The courses are designed so that we won’t forget the terms and skills we learnt throughout time. Average (0-5) Questionnaire item I Strongly Disagree (1) distance education programs The majority of the students stated that courses are not designed to promote permanent learning of concepts and skills, that they are not sufficient in satisfying the needs and that activities which will promote group work and collaboration are not used during course designing. It is evident that students are irresolute on issues such as the courses being arranged from concrete to abstract and from easy to complex, the courses being up-to-date, the comprehensibility of the course language and the difficulty in exploring the online course setting. When qualitative findings stated by the students on the Suggestions section of the questionnaire are considered, it is evident that the rote learning structure of distance education, content insufficiency, the importance of communication among students and the need of improvement in course designing were emphasized. Some of the student opinions stating that the educational system motivates rote-learning and that learning is not permanent due to this are given below: It is not good because knowledge-based questions are asked, education is based on rotelearning. European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 159 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS The system would be more educational if there were practice-based activities rather than a rote-learning system. Various opinions related to content insufficiency are given below: I find course presentations insufficient. The instructors just read the slides they upload to the system. That’s why there are no reasons to participate in the courses. I would prefer a more up-to-date course presentation reinforced with practical examples. A student stated the importance of increasing the interaction between students as: I believe that communication and socialization between students positively affects education. One other student suggested that the course design should be improved and emphasized the need for creating an eye-pleasing course environment When these student suggestions and comments are considered, it is evident that there is a need for the distance education programs to be eluded from rote-learning structures, to promote interaction between the students, the course contents to be enriched and the course designs to be improved. 3. Findings on student evaluations on the teaching and learning process When the satisfaction questionnaire conducted on the students is considered, it is evident that student opinions on the teaching and learning process are generally negative. It was observed that the average of the answers given to six of the 10 questionnaire questions for this research question was between 1.81-2.60, in other words at low level; and the average of the answers given to the four questions was between 2.61-3.40, in other words at moderate level. Findings on student evaluations are given on Table 4: European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 160 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS Table 4: Findings on student evaluations on the teaching and learning process in 2,56 13. Various instructional methods are applied. 2,50 14. The methods applied in course presentation are convenient for distance education. 3,05 17. We can actively participate in lessons. 2,52 21. The examples given during the lessons are sufficient. 3,02 23. Learning activities promote the interaction we will establish between the content (our active participation in the courses, being able test ourselves at the end of the lesson, receiving a feedback on the test results from the system etc.). 24. Learning activities encourage the interaction that we will establish with our peers receiving distance education. 2,92 81 15,1 175 32,8 136 25,5 125 23,4 134 25,1 72 13,5 156 29,2 75 14 84 98 18,3 120 22,5 144 27 141 26,4 146 27,3 92 17,2 137 25,7 93 17,4 102 160 30 127 23,8 143 26,8 152 28,5 142 26,6 164 30,7 138 25,8 165 30,9 168 139 26 82 15,3 73 13,6 80 15 80 15 145 27,2 69 12,9 144 27 127 55 10,3 29 5,4 37 6,9 35 6,5 31 5,8 60 11,2 33 6,1 56 10,5 52 % 15,7 19,1 31,5 23,8 9,7 f 2,49 I Strongly Agree (5) 2,54 I Agree (4) 2,48 f % f % f % f % f % f % f % f % f I Partly Agree (3) 2,97 I Disagree (2) 4. The instructor introduces himself and we can access online the introduction website of the instructor. 5. We can easily access the course teacher whenever we like. 11. We get instant, comprehensive and constructive feedback during the learning process. 12. Our individual needs are taken into consideration during the learning process. I Strongly Disagree (1) Questionnaire item Average (0-5) distance education programs 145 142 145 69 32 % 27,2 26,6 27,2 12,9 6 It was observed that the majority of the students stated that the student-teacher relationship is weak, the feedback received during the learning process is insufficient, individual needs are not taken into consideration during the learning process, active participation in courses is not promoted and learning activities do not encourage interaction among peers (between the 1.81-2.60 range). It was observed that the participants were irresolute on issues related to the teachers introducing themselves, accessing the web pages that introduce the teachers, the convenience of course methods with distance education, the sufficiency of the examples and the learning activities promoting interaction with the course contents (between the 2.61-3.40 range). European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 161 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS When student answers given to the open-ended question are considered, it is evident that 31 students complained about the synchronous courses hours and the obligation of attendance; 16 participants expressed the problem of lack of communication with the teacher; 7 participants complained about the insufficiency of the vocational and applied courses and 5 participants demanded co-education. Lack of examples related to the subject, presenting the course too quickly and canceling various courses were stated as reasons for failing to actively participating in courses. Various student opinions on synchronous courses are given below: The fact that the lesson hours are arranged by the school, the obligation to participate in live lessons, that the lesson curriculum of the first term changes in the second term, that the petition I signed about this was not taken into consideration and responded to are all bad factors. I have a timing problem in attending live lessons, the person follows the courses when available in distance education but the system through live lessons creates a difficulty or distance education students. During live lessons, students have the opportunity to watch the lesson again and the unnecessary questions asked there negatively affect other students to focus on the subject. I didn’t have any problems with the distance education program during the first term. When I did have problems, the authorities would solve them or inform us by giving us the required information. We don’t know why but the system changed this term. Listening to the lessons has become obligatory. Because everyone works in different professions they chose to benefit from this kind of a system. That’s why course participation shouldn’t be obligatory, live lessons should be like last term, once every week or every month and they shouldn’t be obligatory. The responsibility should be laid on the students when the courses are not obligatory. I believe that the target efficiency will not be obtained from any work that is performed through force. Various student opinions emphasizing the lack of communication with their instructors are given below: Each course should have counseling hours, we cannot access the instructors during the lessons due to completing the lesson on time. We can at least ask questions during counseling hours. European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 162 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS Plans should be made to increase awareness so as to facilitate communication with the instructors. The question, suggestion or opinion should be conveyed to the instructors despite being a distance education system. I request the instructors to actively use their e-mails. The information given by the instructors were not enough before the exams. But this is not true for all instructors . Student opinions on the insufficiency of vocational and applied courses and coeducation are given below: We need to take more vocational, face-to-face and applied courses related to our department. I believe that the majority of the courses we take online should be vocational courses. I would have preferred taking vocational courses as applied courses. I would have liked it if applied courses were increased. We want the formal training right for obligatory courses. Free formal course quotas should be offered as an alternative in cases of demand when distance education programs are charged. This will offer an equality of opportunities both economically and also to students who are interested in the course. When student suggestions and opinions are considered, it is evident that the distance education programs should be improved with regards to the communication made with the instructors, feedbacks given during the learning process, individual needs being considered throughout the learning process, use of various learning methods, promoting active student participation during lessons, the interaction between learning activities and the content, encouraging interaction with peers, the synchronous courses being obligatory and the applied courses. 4. Findings on student evaluations on the technology being used It was observed that student evaluations about the technology being used in distance education programs are at moderate level. It was observed that among the 3 answers given to this questionnaire question, the average of one answer was between 3.41-4.20, in other words at high level; and that the average of the other two answers was between European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 163 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS 2.61-3.40, in other words at moderate level. Findings on student evaluations are given on Table 5: Table 5: Findings on student evaluations concerning technology in I Disagree (2) I Partly Agree (3) I Agree (4) I Strongly Agree (5) 3. We can access the current information and notices through the web portal. 25. The technology selected for distance education is convenient and fully accessible for us (we can access the library, course resources and learning materials from the campus, the dormitories and our houses). 27. Every related current information and notices are conveyed to us through mobile means. I Strongly Disagree (1) Questionnaire item Average (0-5) distance education programs 3,49 f % 3,13 f % 36 6,7 74 13,8 47 8,8 78 14,6 134 25,1 164 30,7 176 33 138 25,8 140 26,2 79 14,8 3,27 f 66 73 138 161 95 % 12,3 13,6 25,8 30,2 17,8 The majority of the students evaluated distance education programs positively with regards to accessing the current information and notices through the web portal. It was observed that they were irresolute concerning the technology being convenient and fully accessible and all the current information and notices being conveyed to them through mobile means. Answers given to the open ended question were observed to tackle technical problems; they emphasized systematical issues, mobile support, audial and visual quality. Various student opinions are given below: I cannot attend to live lessons due to technical problems and I am concerned about failing in the class. I cannot print out the course notes given as course resources due to the system they use. We are not given an attendance grade when we attend live lessons through mobile phones. We had a course following system through mobile phones, we still do but we can’t listen to lesson records on mobile phones. European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 164 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS The issues stated under technical problems were the lessons being uploaded to the system too late, the system freezing during the exams, failing to download the materials through the internet and wrong course and instructor assignations. When these suggestions and opinions of the students are considered, it is evident that there is a need to improve technology so as to minimize systematical problems, to offer mobile support and to take into consideration sound-image quality. 5. Findings on student evaluations on instructional materials The average of the two questionnaire items, which aimed at revealing student evaluations on instructional materials in distance education programs, was observed to be between 2.61-3.40, in other words at moderate level. Findings on student evaluations are given on Table 6: Table 6: Findings on student evaluations concerning the instructional materials of distance designed according to multimedia designs (audio, 3,24 video, convenient technology). 28. The content and order of the materials are interesting. 2,91 69 167 164 I Strongly Agree (5) I Agree (4) 58 I Partly Agree (3) f 18. The course material used during the lessons is I Disagree (2) Average (0-5) Questionnaire item I Strongly Disagree (1) education programs 75 % 10,8 12,9 31,3 30,7 14 f 78 103 185 120 47 % 14,6 19,3 34,7 22,5 8,8 It is evident on Table 6 that the majority of the students are irresolute about whether or not the materials used in the courses are designed according to the multimedia design and whether or not the content and order of the learning materials are intriguing. When the qualitative findings written by the students under the Suggestions section are considered, it is evident that 16 students complained about the instructional materials being insufficient and 13 students requested printed materials/books. Opinions of some students expressing the problem of insufficient materials are given below: I would have preferred the educational materials within the system to be more diversified and rich in content. European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 165 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS The materials and presentations of some of our courses are not current and pleasing. More effective materials and resources can be used. Some students who requested printed materials/books expressed these statements: We want the books related to the courses to be given to students in return of paying their costs. We sometimes encounter problems on the internet. It would be better if we had resources. I want the printed materials to be distributed to the students at the beginning of the term. I want books to be given. It is difficult and almost impossible to study on the computer, so I request books. I want books to be given the next term, please. Other problems related to instructional materials were insufficiency of study questions, wrong writing/inconsistency in various subjects, that course notes are not clear and comprehendible, that the course materials are uploaded late or incompletely, failing to access various course materials through the web portal and the materials being up-todate. When these suggestions and comments of the students are considered, it is evident that they suggested the instructional materials to be diversified and enriched, the materials to be easily accessed, the materials to be up-to-date and the distance education programs to be improved in providing printed materials/books. 6. Findings on student evaluations on the support provided It was observed that student evaluations on the support provided to students in distance education programs are generally negative. It was observed that the average of the answers given to three of the four questionnaire questions for this research question was between 1.81-2.60, in other words at low level; and the average of the answers given to the remaining questions was between 2.61-3.40, in other words at moderate level. Findings on student evaluations are given on Table 7: European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 166 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS Table 7: Findings on student evaluations concerning the support provided in 16. Our question and suggestions are taken into consideration. 29. We are informed about the program on issues such as acceptance conditions, tuitions, course books, technical equipment, discipline conditions and student assistance services etc. and related trainings are given to us when we request. 30. We are able to access to suitable guidance services including psychological, professional and social guidance. 31. We can easily access student services that assist our learning. 2,91 2,52 I Strongly Agree (5) I Agree (4) I Partly Agree (3) I Disagree (2) Average (0-5) Questionnaire item I Strongly Disagree (1) distance education programs f 99 87 167 121 59 % 18,5 16,3 31,3 22,7 11 f 142 137 136 83 35 % 26,6 25,7 25,5 15,5 6,5 f % f 2,57 % 2,47 151 28,3 115 21,5 119 22,3 149 27,9 152 28,5 164 30,7 79 14,8 65 12,1 32 6 40 7,5 The majority of the students stated that the information and training given about the program is not sufficient, that they cannot access the appropriate guidance services and that they cannot easily access student services. It is evident that the students were irresolute about their questions and suggestions being taken into consideration. When the qualitative findings written for the open-ended question are considered, it is evident that support is not provided for student problems and that feedback is not given to the students. Examples of student opinions on the support given for problems and the lack of feedback are given below: We wrote petitions about our problems but they weren’t taken into consideration, we didn’t even get a reply. We encountered warnings that are not even appropriate for adults. We don’t even have a responder to convey our complaints to, we send e-mails to access them but there is no one to consider them and give us a reply. We don’t have an official to convey our complaints to and who will resolve them, even there is a person, we don’t know him. It is a totally unjust order. Rules are made without giving any information. Out student rights are not defended by any parties. European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 167 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS When student suggestions and opinions are considered, it is evident that improvements should be made on informing students about the distance education program, providing appropriate guidance and student assistance services and taking student questions and suggestions into consideration. 7. Findings on student evaluations on the assessment-evaluation dimension It was observed that student evaluations on the assessment-evaluation processes in distance education programs are generally negative. It was observed that the average of the answers given to three of the 6 questionnaire questions for this research question was between 1.81-2.60, in other words at low level; and the average of the answers given to the remaining three questions was between 2.61-3.40, in other words at moderate level. Findings on student evaluations are given on Table 8: Table 8: Findings on student evaluations concerning assessment-evaluation in distance 2,56 I Agree (4) I Strongly Agree (5) 83 15,5 97 18,1 139 83 15,5 92 17,2 127 163 30,5 160 30 158 158 29,6 125 23,4 82 46 8,6 59 11 27 % 26 f % f 2,47 % f 2,52 % 2,99 I Partly Agree (3) f % f 2,91 % f 3,00 I Disagree (2) 6. There are assessment methods, homework feedbacks and homework explanations in every course. 32. Evaluation processes are conducted appropriately and on time, they respond to our needs. 33. Along with written exams (mid-term and final), alternative evaluation methods (written and oral tasks, presentations, portfolios, self-evaluations, peer evaluation, group work etc.) are also used. 34. Learning evaluation is done according to transparent, reliable, realistic and valid criteria. 35. Evaluation is carried out both during the learning process and also after the learning process. 36. There is a fair and effective system concerning our complaints on evaluation results. I Strongly Disagree (1) Questionnaire item Average (0-5) education programs 81 15,1 141 26,4 143 26,8 23,8 29,6 15,3 5 92 17,2 160 30 147 27,5 163 30,5 168 31,5 142 26,6 144 27 37 6,9 68 12,7 53 9,9 27 5 33 6,1 According to the findings, the majority of the students believe that alternative evaluation methods are not used, the process evaluation is not sufficient and there isn’t a fair and effective system for their complaints related to the evaluation results. It was observed that they were irresolute about explanation given to the assessment methods, European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 168 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS homework feedbacks and homework, the evaluation processes being conducted on time and appropriately, getting a response about their needs and learning being evaluated according to transparent, realistic, reliable and valid criteria. When student answers given to the open-ended question are considered, it is evident that there are complaints about the exams and homework being too difficult and the fact that mid-term exams have little contribution to the grades. Various student opinions on this issue are given below: I want the exams to be easier. The exams are too difficult. They are even more difficult for our hardworking friends. The instructors are doing everything they can for us to fail. It is too much for the passing grade to be 70 and the final exam being 80% effective in passing. The passing grade and criteria should be arranged equally. The passing grade can be reduced to 60 like in other universities or relative distribution can be applied. The student passing the course depends on the final exam, we pass if we get 80 from the final exam or else we fail. Success and productivity is not possible in a system like this, that’s why we need a sound and logical change in the system. When questionnaire findings are considered, it is evident that the problem of not being graded for attending the system was expressed; the students complained about the exams being on the same date, the questions being asked from misrelated subjects and other students cheating. Various students stated their opinions on cheating in the exams as: It is difficult to avoid the feasibility of sharing information through various other blogs, that’s why there is no permanent learning. I don’t find mid-terms results fair, the time given is too long and because the questions are the same as the ones on the presentations, it is easy to cheat. Midterm and final exams should be carried out in the classroom. I believe that a fair evaluation is not made in the present system. No measures have been taken about cheating and plagiarism. European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 169 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS The problem of failing to access optical forms and exam booklets was stated as below: I can’t access the exam questions and see where I made the mistakes. Various other problems were stated about the assessment-evaluation system directing the students towards parrot fashion, about homework evaluations not being sturdy, the exams being conducted in places far to places of residence, the questions being incorrect, giving insufficient information about which subjects the questions will be asked, reducing the number of quizzes and not carrying the finals through the internet. When student suggestions and opinions are considered, it was suggested that alternative evaluation methods should be included in distance education programs, complaints about evaluation results should be considered, the rates of mid-term exams and homework on the grade should be increased, the problem of cheating should be resolved, questions that direct to parrot fashion should be avoided, optical forms and booklets should be open to student access after the exams. Discussion and Conclusion The purpose of this study was to identify student evaluations and satisfaction levels concerning the pedagogical dimension of distance education programs. Findings suggest that students find it necessary to improve distance education programs in course design and content, the teaching and learning process, educational materials, the support provided and on assessment-evaluation. According to the study findings, the students requested the content of distance education programs to be enriched and the course designs to be improved. In addition, it was observed that students found the interaction among students insufficient and suggested the communication with the instructors to be improved and to receive feedback during the learning process. Li (2009), who emphasized the importance of communication in distance education settings, stated that student-teacher and studentstudent interaction in distance education settings are very low and discussed the negative effects of this. Wetzel (2009) stated that student communication with the instructor in distance education is crucial for their course achievement. Improvements should be made in content and course designs in distance education programs and measures should be taken to better conducted student-student and student-teacher interactions. European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 170 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS It is crucial to various instructional methods and to enable active student participation throughout the educational process. However, according to the findings, students find distance education programs insufficient from these characteristics. Similarly Pallof and Pratt (1999) stated that student interaction is not taken into consideration much in online learning settings, a discussion setting where students can actively participate is not created because open-ended questions are asked, instant feedback is not given and that the learning process is not student-oriented but rather instructor oriented. Schrum (1999) also stated that online learning settings where students can actively undertake roles should be established. It was observed that the instructional materials of distance education programs are insufficient; and that the students suggested the materials to be diversified and enriched, the materials to be easily accessed, the materials to be up-to-date and printed materials/books to be provided. Wetzel (2009) underlined that it is a big issue that distance education students use only the resources that are provided for them and emphasized that it is crucial for institutions to offer all their research opportunities and online resources to their students. Armutlu and Akçay (2012) underlined the importance of distance education instructors and editors uploading their own resources or the resources they gather by reviewing the literature to the system; and stated that educational resources and materials should be appropriate for distance education and should be convenient for publishing online while being created. The students stated that they want to be informed about the distance education programs; and demanded appropriate guidance and student assistance services to be offered and their questions and suggestions to be taken into consideration. Similarly Can (2012) stated that it is important to inform the students about the distance education program and the registration procedures and that they can even be trained about these issues if necessary. Galusha (1997) stated that assistance to distance education students should not be avoided while planning distance education programs and emphasized that students need a consultant and academic guide who can assist them in cases of a problem and who can help them passing their courses on time. Students who stated that evaluation processes are not conducted appropriately in distance education programs suggested that alternative evolution methods should be included, process evaluations should be made, complaints about evaluation results should be considered and the rates of mid-term exams and homework on the grade should be increased. ”akioğlu and Can emphasized in a similar study that course achievements of students should be evaluated through different methods in distance education programs and stated that a change in the exam system is required. Oliver (2015) asserted that student achievement on attaining their goals can be identified in European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 171 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS different ways through alternative evaluation methods. Kaya and Tan (2014), who underlined the importance of process evaluation, stated that process evaluation in distance education can be conducted through formative tests on a website and feedbacks can be given to the students and added that the students can shape their following studies, restock their deficiencies on the subject and get rapid feedback from large student societies. That the weighted mean of the passing grade is calculated from supervised exams was stated as a problem in assessment-evaluation processes; and it was underlined that the percentage score calculations of assessment-evaluation processes and their effects on the passing grade should be restructured. Similarly, about student evaluations, Çağıltay criticized that the passing grade is given mainly from the supervisory exams and stated that this sets a drawback for alternative evaluation methods unique to distance education settings and causes evaluations to be conducted through classical methods such as in the classroom setting. It is surprising that students are complaining about the cheating done in exams and want a solution to be found cheating. Parallel with this finding, Anderson, Cain and Bird (2005) stated that online exams are not reliable, that there is a possibility for the students to cheat and that it is too difficult to control whether or not the students themselves attended the exams. It should be taken into consideration that self-evaluation has a crucial effect in improving the quality of distance education programs. Detecting the problems by collecting statistical data related is too the use of the program and using these data for making improvements will increase the quality of the education offered. Evaluating the students’ opinions and satisfaction levels in regular intervals and improving the program based on these evaluation findings will have positive contributions is too the distance education process. It is expected that studies is too this study, which aimed at evaluating distance education programs pedagogically through student opinions, examining various other dimensions of distance education programs via different shareholders will have major contributions is too the literature. References 1. Anderson, H. M., Cain, J. ve Bird, E. (2005). Online course evaluations: Review of literature and a pilot study. American Journal of Pharmaceutical Education, 69(1), 34-43. 2. Armutlu, H. & Akçay, M. (2012). Uzaktan Eğitimde Kaynak Yönetimi [Resource Management in Distance Education]. XIV. “kademik ”ilişim Konferansı ”ildiri European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 172 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS Kitapçığı [“cademic Computing Conference Proceedings Handbook]. st edition . Ed: M. “kg(l, U. Çağlayan, E. Derman, “. 5zgit, Ş. G(ven, K. Kahraman). Usak University. 3. ”akioğlu, “. ve Can, E. . “çıköğretimde 5lçme ve Değerlendirme: Problemler ve Öneriler [Measurement and Evaluation in Public Education: Problems and Suggestions]. Uluslararası Yükseköğretim Kongresi:Yeni Yönelişler ve Sorunlar [International Higher Education Congress: New Trends and Issues]. 27-29 May 2011, Istanbul: Turkey. . “nket geliştirme [Survey development]. Türk Eğitim 4. ”(y(közt(rk, Ş. Bilimleri Dergisi [Turkish Journal of Educational Sciences], 3(2), 133-151. 5. Can, E. (2012). “çık ve Uzaktan Eğitimde “kreditasyon Yeterlilik Düzeyinin İncelenmesi [Investigation of the Open and Distance Education “ccreditation Proficiency Level]. Unpublished PhD Thesis. Marmara University, Institute of Education Sciences, Istanbul. 6. Cohen, L., Manion, L., & Morrison, K. (1997). Methodology of educational research. Athens: Ekfrasi. 7. Çağıltay, K. . Uzaktan Eğitim: ”aşarıya Giden Yol Teknolojide mi, Yoksa Pedagojide mi? [Distance Education: Does the The Road to Success Technology, the Pedagogical Or is it?]. Retrieved from http://ocw.metu.edu.tr/file.php/118/Week10/Cagiltay.pdf on 16.07.2015. 8. Galusha, J.M. (1997). Barriers to learning in distance education. Interpersonal Computing and Technology: An Electronic Journal for the 21st Century, 5(3-4), 6-14. 9. Karasar, N. (1999). ”ilimsel “raştırma Yöntemi: Kavramlar, İlkeler,Teknikler [Research Methods: Concepts, Principles, Techniques]. Ankara: Nobel Publications. 10. Karataş, S. . Deneyim eşitliğine dayalı internet temelli ve yüz yüze öğrenme sistemlerinin öğrenci başarısı ve doyumu açısından karşılaştırılması [Comparison of internet-based learning systems and face to face learning systems in terms of student achievement and satisfaction]. Unpublished PhD Thesis, Institute of Education Sciences, Ankara. 11. Kaya, Z. (2002). Distance Education. Ankara: PegemA Publishing. 12. Kaya, Z., & Tan, S. (2014). New Trends of Measurement and Assessment in Distance Education. Turkish Online Journal of Distance Education, 15(1), 206-217. 13. Lentell, H. (2012). Distance Learning in British Universities: is it possible?. Open Learning: The Journal of Open, Distance and e-Learning, 27(1), pp.23-36. 14. Li, X. (2009). Review of distance education used in higher education in China. Asian Journal of Distance Education, 7(2), 22-27. European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 173 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS 15. Middlehurst, R., Woodfield, S., (2004). International Quality Review and Distance Learning: Lessons From Five Countries. Centre For Policy and Change In Tertiary Education, University of Surrey, CHEA Occasional Paper, USA. 16. Oliver, E. (2015). Alternative assessment for effective open distance education.Unpublished Master Dissertation, University Of South Africa 17. Palloff, R.M. & Pratt, K. (1999). Building learning communities in cyberspace: Affective strategies for online classroom. San Francisco: Jossey-Bass 18. Patton, M. Q. (2014). Nitel araştırma ve değerlendirme yöntemleri. . baskıdan çeviri). (M. Bütün ve S. B. Demir, Çev.) Ankara: Pegem Academy. 19. Rivera, J.C., McAlister, M.C., & Rice, M.L. (2002). A comparison of student outcomes and satisfaction between traditional and web-based course offerings. Online Journal of Distance Learning Administration, 5(3). 20. Rumble, G. (2001). The costs and costing of networked learning. 21. Sanders, J. R. ve Nafziger, D. N. (1976). A basis for determining the adequacy of evaluation design. Occasional paper, 6, Kalamazoo: Western Michigan University Evaluation Center. 22. Schrum, L. (1999). Trends in distance learning: Lessons to inform prac-tice. In Branch, R.M. & Fitzgerald, M.A (Eds.). Educational media and technology yearbook, 24, 11-16. 23. Thomas, R. M. (1998). Conducting educational research: A comparative view. West Port, Conn: Bergin & Garvey. 24. Uşun, S. . Uzaktan Eğitim [Distance Education]. Ankara: Nobel Publishing. 25. Uşun, S. . Eğitimde Program Değerlendirme, Süreçler Yaklaşımlar ve Modeller [Curriculum Assessment, Process, Approach and Models].“nkara: “nı Publishing. 26. Wetzel, D. R. (2009). Top 10 Distance Education Programs. Retrieved from https://suite.io/david-r-wetzel/1e1520x on 15.04.2015. 27. Worthen, B. R., Borg, W. R. & White, K. (1993). Measurement and evaluation in the schools. NY: Longman Publishing Group. European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 174 Betül Tonbuloğlu, Aysun Gürol – ANALYSIS OF DISTANCE EDUCATION STUDENTS’ OPINIONS AND SATISFACTION LEVELS OF THEIR PROGRAMS Creative Commons licensing terms Authors will retain the copyright of their published articles agreeing that a Creative Commons Attribution 4.0 International License (CC BY 4.0) terms will be applied to their work. Under the terms of this license, no permission is required from the author(s) or publisher for members of the community to copy, distribute, transmit or adapt the article content, providing a proper, prominent and unambiguous attribution to the authors in a manner that makes clear that the materials are being reused under permission of a Creative Commons License. Views, opinions and conclusions expressed in this research article are views, opinions and conclusions of the author(s). Open Access Publishing Group and European Journal of Open Education and Elearning Studies shall not be responsible or answerable for any loss, damage or liability caused in relation to/arising out of conflict of interests, copyright violations and inappropriate or inaccurate use of any kind content related or integrated on the research work. All the published works are meeting the Open Access Publishing requirements and can be freely accessed, shared, modified, distributed and used in educational, commercial and non-commercial purposes under a Creative Commons Attribution 4.0 International License (CC BY 4.0). European Journal of Open Education and E-learning Studies - Volume 1 │ Issue 1│ 2016 175