How useful did you find the formative evaluation exercise
with your colleagues or students that you had to implement last week? What did you learn? What changes could you
make to your prototype in the light of the feedback received from peers, your
class participants and facilitators? This will feed into your prototype
presentation.
Show
the activity and explain what informed your design decisions, by answering the
following questions:
- How did I go about exploring what I needed to do?
- How did I design and develop my learning activity?
- How did I formatively evaluate my prototype learning activity?
It has been an interesting journey up until this point. My prototype has evolved forward a little, regressed back a little, changed a little, and the progressed again. I have not managed to address student in terms of feedback, but colleagues have been supportive. However, with colleagues, I spend most of the time explaining the purpose and uses of the tool, and the idea of using technology in higher education, that not too much specific feedback has been enough to stimulate major (or minor really) changes to my prototype. More information exchanged regarding the use of this prototype in other challenges, which is great for moving beyond this course, but not too useful for the immediate future!
ReplyDeleteI have made some changes based on facilitator feedback, mostly around the use of theoretical starting points. This has occupied much of my reading time, and I am still not sure about some of the concepts. This is proving to be my biggest challenge. Furthermore, there is no literature I can find incorporating my prototype idea, so even more difficult.
So the 3 questions to answer...well, I have been exploring what I needed to do by self reflection in terms of student challenges. I then consulted what literature and other internet resources I could find to help with the challenge, and evolved a potential solution from there. The design and development has been guided by thoughts while running in the mountains (no, really, running is my best thinking time - I was actually half why up Helderberg mountain when the prototype struck me), and further developed based on a fraction of literature insight, and a whole load of facilitator assessment, which is really where the formative evaluation of this activity begins and ends.
It has been an interesting journey up until this point. My prototype has evolved forward a little, regressed back a little, changed a little, and the progressed again. I have not managed to address student in terms of feedback, but colleagues have been supportive. However, with colleagues, I spend most of the time explaining the purpose and uses of the tool, and the idea of using technology in higher education, that not too much specific feedback has been enough to stimulate major (or minor really) changes to my prototype. More information exchanged regarding the use of this prototype in other challenges, which is great for moving beyond this course, but not too useful for the immediate future!
ReplyDeleteI have made some changes based on facilitator feedback, mostly around the use of theoretical starting points. This has occupied much of my reading time, and I am still not sure about some of the concepts. This is proving to be my biggest challenge. Furthermore, there is no literature I can find incorporating my prototype idea, so even more difficult.
So the 3 questions to answer...well, I have been exploring what I needed to do by self reflection in terms of student challenges. I then consulted what literature and other internet resources I could find to help with the challenge, and evolved a potential solution from there. The design and development has been guided by thoughts while running in the mountains (no, really, running is my best thinking time - I was actually half why up Helderberg mountain when the prototype struck me), and further developed based on a fraction of literature insight, and a whole load of facilitator assessment, which is really where the formative evaluation of this activity begins and ends.
Dear Kris, your presentation this morning was quite detailed and contained a lot of useful information around blogs. Thanks for that and the way you thoughtfully participated throughout the course. To my mind this experience of (painfully often) working through a kind of workflow or process, and reflecting on it, really switches one on to what might be possible in future. I also love to run and Mountain bike, and can appreciate your hillside insigths :) All the best with implementing your prototype, and let us know how it went. See you at Heltasa!
DeleteThis comment has been removed by the author.
ReplyDeleteFormative evaluations feedback
ReplyDeleteFor my formative evaluations I chose my colleagues to test the idea out on as the students are away for exams. I was really surprised by the results. It has made me think a lot about my prototype and of better ways of encouraging it.
For the formative evaluation, I chose to do a five question questionnaire just to present the idea and see what the general idea was around my prototype. As our department is small it was not that difficult for me to hand them the questionnaire and have them return it to me by the end of the day. The five questions asked were as follows:
1. Do you have a face book account? Yes or no
2. How often do you use face book? Often or hardly
3. What are your ideas on using face book as an educational tool?
4. Would you consider using face book as an educational tool? Yes or no
5. Comments or concerns about using this tool for educational purposes.
The results are as follows:
1. 86% yes and 14% No
2. 43% often and 57% hardly
3. Ideas that were generate was using face book for assignments, comments, reminders to the class etc. many stressed that it would need to be monitored very carefully.
4. 43% yes and 57% no
5. Comments: many were not comfortable using this tool as they have never seen it being used before and therefore were not sure of its success rate. Many stressed that it was too much of a social tool and due to this affordance they would not trust the tool. The few that were willing to try the prototype were confident as they were current face book users and know how to control this tool. They all were in consensus that this method of teaching will have to be strictly monitored and controlled to prevent chaos.
What did I learn?
• I leant that although my prototype will immediately spark interest with the students it might create a bit of panic with the lecturers as they are not use to this tool. Proper training needs to be done in order for this tool to work.
• I learnt that using this tool over long periods of time will not be wise as then learning world becomes mixed with social world and this often leads to chaos.
• Lastly the part that stuck out at me was the need for this tool be controlled or monitored in order to succeed. I will need to implant really strict guidelines if I want to use this tool in teaching and learning. This will be the major change that I will need to add to my prototype
thank you for the wonderful and educational jounery.
Dear Khalida. The formative feedback is indeed interesting but also voice concerns expected from colleagues who are not necessarily comfortable or familiar with such a tool. I agree that one needs to be careful in implementing such a tool in teaching - the challenge remains to distinguish between the social realm and learning environment. Often clear expectations and discussion beforehand with students could lead to a mutual agreement in the use of such a tool in the classroom. Good luck with tomorrow.
DeleteSonja
From Taryn:
ReplyDeleteHow did I go about exploring what I needed to do?
I engaged with colleagues and students to get input on what can be done to improve the engagement between students and between lecturers and students in initiating, implementing and completing a research project. Many aspects were raised however I realised, after considering feedback received on my reflections, that I needed to start small.
How did I design and develop my learning activity?
I based my design and development on the the underpinning approach that students learn from one another. On the concept of Communities of practice - 'groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly.'
I then asked myself how I would like to do this. A few things emerged in doing this and it helped me to decide on the activity.
• Be online
• Be a closed site with access to all students working on their research projects, recent graduates and relevant faculty
• To be able to have key resources available – this includes guidelines for projects and also links to useful websites such as Ethics websites, reporting guidelines, etc
• Place for students to post questions related to their projects and to get responses from fellow students / lecturers/recent graduates – this can be asynchronous or synchronous
• Place where recent graduates can share their stories – this can be in the form of a words / screencast / podcast
• Place where we can have online synchronous sessions where we discuss issues related to their research projects
• To receive notification by email when there is a new question/comment
I initially wanted to start a blog but then opted for a learning management system using Edmodo.
How did I formatively evaluate my prototype learning activity?
I asked peers and recent graduates to test the LMS and found their feedback quite useful. This ranged from comments on the content to comments on the functionality.
Dear Taryn! First of all congratulations to your prototype presentation. Its amazing to see who far you have progressed in these few weeks. And to actually have created the Edmodo site and have students interact already. I am very impressed. I think your idea of supporting your learners through such a shared space is brilliant. I wish you all the best! And hopefully you will consider presenting with us at HELTASA!
DeleteFrom Waldo Part 1
ReplyDeleteI learned that I can support the Blog with a link to a Facebook group to inform students of the class blogspot they have to read to complete an assignment and or task. On the blogspot I will post an event which is augmented by articles, and or you tube video which they have to read. I was thinking in the beginning to use a "polleverywhere" to establish if they have web-enabled phones, would like to join a class facebook group and blog spot and if they read current events. Equally so my peers argued that can use a normal e-mail message to inform the students on assignments and tasks.
I have heard that if a LMS is used that I can manage (mark) a blog(s) by using a rubric or issue a rubric on the blog to facilitate the students' assignment and give structure to the blog in a step by step process. This will be in a formative way. Only two students’ out of seven responded to questions I posed on using facebook and blogging. Students just finished writing exams and the responses are slow.
Students do how ever have concerns about availability of airtime, blogs being stagnant, reception at the college and networks not up and running. I therefore need to take this inconsideration and have a back-up plan which in this case will be their ultimate reflection on the task where they have to hand in a hard copy of the task and do a 5 min presentation that will be peer assessed.
67% of the lecturers are ok with using facebook to inform students of assignments and 100% supports the idea of using a blog. I do have however some concerns or comments which are the same to the class participants of this course.
How did I go about exploring what I needed to do?
I basically had to assess what resources I have to my disposal to facilitate students to read more and make my classes interesting so that I can also align myself in their “world” I consulted literature which augments the strategy I want to follow.
From Waldo Part 2
ReplyDeleteHow did I design and develop my learning activity?
It was decided upon based on assessing and reviewing different tools as a practical means to better facilitate and coordinate my students. By asking questions to get buy-in and have a participatory approach with the students and peer reviewing from fellow lecturers on the ideas I want to use. The design is to create a blog. On the blog is a statement or comment. The statement or comment is augmented with articles and a video clip that the students must read and watch. The activity asked of them is then to divide into 5 groups of 20 students and in these 5 groups, divide into smaller groups of 5-6 students. The 5 big groups of 20 students need to reflect what they have read and understood. The lecturer follows the group blogging to monitor if they are on the right track and give guidance if needed. Then the process starts of discussions, debate and peer review. The rubric used for this exercise is formative and used as a framework to guide the student /group in the steps they need to take that will lead to summative assessment. In essence my rubric is the step by step instruction in bullet format what they need to do as a large and small group which they also need to read carefully.
The group of 5 students’ chooses a topic which they then have to present in hard copy and do an oral presentation. The hardcopy is formatively assessed by the lecturer and the oral presentation assessing via their peers and lecturer. The later is facilitated by the lecturer.
How did I formatively evaluate my prototype learning activity?
By taking part in a reflection session by peers and facilitators to highlight the pros and cons of different tools and the affordances it can bring to a facilitator and the students. I also used questionnaires to get feedback if the tools mentioned above will be conducive or act as an indirect conduit that can lead to comprehensive reading skills and at the same time lead to better student-teacher relationships, encourages active learning, creativity, problem solving, critical thinking and allows for greater flexibility in teaching and learning.
Dear Waldo, I can see that there is a lot of thought that has gone into your prototype. But sometimes there is also only so much you can design and plan without really doing it. Many questions that you raise or that come to my mind when I read your blog, might be only answered once you implement your ideas. I wonder if it wouldnt be useful to pilot your intervention with a few students first. Just for you to see how students respond to the task and to tweak and fine tune the process. Your intervention requires quite a few steps and some of those steps you will only really understand when you do it with your students. I also feel that the intervention is still quite teacher led and controlled. Maybe that's what you need with your students, but I wonder if you couldnt allow your students a bit more freedom, e.g. in choosing topics they would like reflect and present about...
DeleteReflection 4
ReplyDeleteFeedback from lecturers, facilitatots and students was a good indication of what is not viable and the problems that had to be sorted out before implementation.
The learning challenges flow one into the other and I tried to address all of this at once which was not possible from the feedback in the online session. I also realised that all of this chain reaction” problems can be addressed with a Learning Management System. The Institute does not have a LMS and the processes were started to have a LMS ideal for our circumstances. On this basis my prototype changed completely and I focused on the learning problem that is affecting the performances of students in this course and a method to overcome it.
With this problem the tablet will be introduced that take the student through a step-by-step process to calculate the answer and interpret it to make a correct decision. The tablet will be used in a lecture setup and all of the proceedings recorded to publish it in a LMS. The students can then download and go through the process and post a question. These questions will be addressed in a lecture and the tutorials will be formalised based on this questions. Problems will be given to groups of two or three with each a tablet. They are allowed to discuss the problem and work out the most reasonable answer and with a Bluetooth connection to the projector; they will explain the steps to the rest of the class to eventually get to an answer and a decision.
reflection 4 part 2
ReplyDeleteMy exploration started by listing all the problems that we (student and lecturer) are faced with in this course. The article of Dabbach and Bannan-Ritland (2005) and the information on the Emerging Technologies Wiki were studied to match it with the problems. Internet restrictions at the Institute, the lack of a Learning Management System and student backgrounds were the basis to eliminate the technologies that are not viable. I initially focused on the tool to address the learning problem and during the second Adobe Connect session I observed that the learning problem should be my focus with the prototype as my tool to address it.
It is expected of students to determine the amount of water and the direction of water flow in a soil with mathematical equations and then make decisions with the management of irrigation. The student is able to calculate the answer by memorising the equation but they are unable to make the decisions as they don’t understand the process to the answer of the calculation. The problem-solving strategy from Heller et. al. (1992) was suggested from a colleague and in the class setup I followed the five-step problem solving strategy with HP tablet. Classes are big and I encourage discussions with other students to enhance their ability to reason the best possible way of addressing the problem and eventually answering the question. With these presentations students got more involved and requested that these files are made available. Various software were used to convert the files to Windows Media Video/Audio files until a facilitator suggested the use of Camtasia Studio. The activity of class discussion and reasoning in class led to exploring this in smaller groups (tutorial sessions with 15-20 students) and allowing the student to explain the process of solving the problem and making a decision afterwards.
In the tutorial, students had an opportunity to reason with one another to eventually come to a solution that they can explain and share with the rest of the group. This was done by giving each group of three to four students a tablet. And after their discussion they connected to the projector and explained to the rest of the group. Students are then evaluated on their discussions in class as the lecturer can move from one group to another and listen to their reasoning. A mark is given when they follow the five step problem solving strategy from Heller et. al. (1992) and make a decision at the end. This activity was evaluated after a scheduled semester test and students were asked for comments with regards to collaborative learning and the use of tablets in the class and tutorial setup. In the class setup student found it very interesting and helpful for the problem solving through a step-by-step process and these files that are available with audio even on a cell phone. They consider the tablet to have a positive effect on their learning with active participation in the tutorial that made them feel important, comfortable and part of the team and in the end know exactly where they stand. They have high expectations from the lecturer to compile groups strategically and facilitate the sessions to ensure that the concepts and basics are used in the correct manner. In certain cases the use of the tablet was too fast and students could not keep track, especially in the tutorial classes.
This comment has been removed by the author.
ReplyDeletefrom Anita:
ReplyDeleteReflection 4
The feedback from the facilitators in the last Adobe Connect session reminded me to make sure my presentation is 'steeped' in pedagogical rationale and theory. That was quite an aha moment for me. I had a glimpse of the big picture of this course, that it's not just about learning the technologies and putting them to use. Our impact can be greater if we consider the reasons for choosing technology based on theories of learning.
It was also helpful to have some components of the theoretical frameworks relevant to my project voiced, such as lifelong learning, revision of previously learned materials and developing graduate attributes. I'm still getting comfortable with the words 'theoretical framework' and it was a relief to see this need not be dense.
Finally, the practical advice was extremely helpful. I was advised to keep my focus tight for this prototype given the time constraints and to check how easily mathematics can be entered in the prototype. I also valued the ideas for increasing student interaction with the prototype.
From Phaddles:
ReplyDeleteHow did I go about exploring what I needed to do?
At the start of the Checet course we were tasked in identifying problem areas evident in our own course. On reflection I decided that for one of the major problem areas in my class is the divide that exists between student that have prior knowledge and those that are beginners to the module. My next step was then to identify the tool best suited to address this gapping divide.
My interest in wikis and how they could be used as a teaching and learning tool was piqued during the presentation given at the 2nd face-to-face session. My approach to the development and design of my prototype from this point pretty much follows a top-down approach in that I first formulated the idea and designed my prototype and only after did I research articles and other literature that would support my prototype. This was a mistake on my part as I later realised while trying to come to grips with the language and terminology used within the Teaching and Learning context.
During the adobe-connect formative feedback session and subsequent discussions held with colleagues, students and the Checet team, as well as the feedback received from my blog posting, I was encourages to continue with this intervention but also asked to address some design flaws. The governing element in deciding on the wiki was the collaboration aspect. I decided against the blog as I was not looking for day to day account of knowledge development but was more looking at interaction between students as well as the change tracking properties of the wiki. Issues concerning peer review and students being negatively impacted by this were raised but I am of the opinion that a structured peer review method with clear guidelines for students to follow when giving comments can minimise this effect. My thoughts on this is that this aspect of student learning is also necessary to develop the skill required in the later years of study and in the workplace where peer review and feedback is a given.
How did I design and develop my learning activity?
Again my design was initially guided by the wiki presentation given at the face-to-face session. I modeled my prototype in a similar way in that I also will be creating a wiki with subdivisions addressing various aspects of the work covered in the course. I will then divide my class into groups and assign each group a section to work on. Students will be supplied with a broad outline of what is expected, which they can use to guide their wiki development but will not prescribe how the final product should look. Online clicker surveys to evaluate draft stages of the wiki development will be used to address some of the issues that were raised with regards to peer review. Groups will also be assigned to comment on a section other than their own and make suggestions for improvement. I initially thought that I would assign individuals to comment on each other’s work but it was suggested I do this through the group structure to reduce the negative impact of the peer review by making the review a little more anonymous.
One concern raised by a colleague was how I would garner the support of those students with prior knowledge and not make them feel disadvantaged by this intervention. It was suggested that I use a knowledge scaffolding that would develop degrees of difficulty. Meaning that perhaps during the developments process I should look at adding more and more difficult components to the section that a specific group addresses, thus also challenging those students.
How did I formatively evaluate my prototype learning activity?
The prototype was formatively evaluated by discussions with colleagues, the Checet team, and students. Concerns raised were critically analysed and further reading also needs to be done to fully support my idea. This will also lead into refinement of my prototype to address these concerns.
My own reflections on the Checet blog also helped to me assess my design, as well as the comments that I received on them.
Hi Phadlie
DeleteAll is now done and dusted, so my last feedback is to congratulate you on a good, and well thought through prototype. I think you came up with a nicely integrated solution to your teaching problem. I did ask about exactly what the students will need to do their teamwork problems on, and this is probably your next step - the detil design of the intervention. Getting good questions, and scaffolding (as your colleague said) will be crucial. Being very clear about exactly you what you want students to do (on a week to week basis - i.e. to create milestones for them) is also crucial.
Let us know it all transpired. See you next week for Blackboard training.
Mhakamuni's Reflection 4 as received by e-mail:
ReplyDeleteCHECET Reflection 4
Yes, it has been a great experience. Looking back on the first day of our face-to-face meeting to the final day of presenting my prototype, leaves me with the feeling of fulfilment. Yes, the formative evaluation has been useful. The feedback I got from the facilitators, lecturers, CHECET colleagues provided me with relevant information to modify my prototype (Black & William, 1998). I used the information to critically assess the lecturers’ needs, my own needs, tools that lecturers have access to, the affordances of the tools, tasks that I designed, diversity of my students and outcomes that I wanted to achieve.
Equally important, the feedback was timely provided and relevant. Before presenting the prototype to the group of lecturers, I tested the prototype with one lecturer whose course is on both Moodle and Blackboard. I realised that some lecturers could not login. In addition, they could not download Java software for the collaborative tool that I wanted to use to get feedback on their experience in Blackboard and Moodle. I realised that I could use Moodle chat and blog tools for the prototype. Moreover, I realised that I should project the statements on the data projector to accommodate lecturers that used cellphones to respond to the statements. I also realised that the web address after submitting the responses was going to confuse the novice users. I used Moodle url to enable lecturers to return to the prototype page after submitting their responses.
Furthermore, the Head of Department’s feedback on the prototype, made to realise that I have to accommodate diverse needs of lecturers. To accommodate diverse needs of the lecturers I designed a pre-survey form in Maties.connect to establish lecturers’ technological level. The pre-survey assisted me give detailed explanation of the activities that they were supposed to do.
The feedback offered suggestions that were within my understanding. Take for an example, Luvuyo’s comment about that Response System promotes surface learning. I reconsidered the outcomes and the goals of my prototype. I realised that I should allow lecturers to first give their individual opinion on individual statement. After submitting their individual response of each statement, then they discussed in pairs. After discussing in pairs, they had to resubmit their opinions. The discussion, chat and blog promoted collaboration among lecturers.
Mhakamuni ... continued Refection 4 (II)
ReplyDeleteThe feedback helped me to re-evaluate the relationship between the lecturers’ needs, the available tools, their technological level, diversity, risks and the activities. Considering all these factors, assisted me to understand the affordances of the response system, chat and blog. When I considered each tool, I realised what I can do with each tool. I married the affordances of response system, chat and blog together to address the lack of time. The chat tool and blog will promote lecturer-lecturer and lecturer-facilitator (me) interaction.
Actually, my aim was to host an online meeting with lecturers whose modules are uploaded in Moodle or Blackboard, using Adobe.connect. The aim of the online meeting was for the lecturers to share screens of tools they have used in Blackboard /Moodle in their courses. I sent Doodle schedule for the lecturers to book time for the online meeting. Only lecturer responded to the Doodle schedule. I realised that I had to change my prototype completely. I realised that I should not try to impose these technologies, but I have to use the technology that they are familiar with. I had to look for Moodle and Blackboard tools that have collaborative tools. I selected Chat and Blog, although they do not have all tools that Adobe.connect have. Because of the diverse nature of the lecturers, I had to meet them face-to-face in a computer laboratory. Funny enough, most of the lecturers that indicated that they would attend my prototype presentation have not yet uploaded their modules in LMS. I decided to use Welsch’s statements to probe their opinion about use of technologies in teaching and learning.
Lecturers were so positive about using cellphones as clickers. I managed to persuade them to use Moodle and Blackboard for day-to-day operation. Lecturers were so excited to realise that we can use Moodle Collaborative tools to interact more without meeting face-to-face. They realised they are not alone who struggle with technologies. In addition, I indicated that we are all learners, learning what can do with these technologies (Veletsianos, 2008)
Thanks to the facilitators for giving me a chance to reflect every week. I managed to identify the gaps between the lecturers’ needs, my own needs, the problem that I wanted to address, activities that I planned and the tools that I wanted to use. I managed to work on my misconceptions in time. The weekly reflection supported me and encouraged me to explore more tools that I could use to meet the lecturers’ needs.
Mhakamuni reflection 4 continued (III)
ReplyDeleteThe clear, detailed constructive feedback scaffolded me to produce a well crafted prototype that I will present on the 6th June. Although I consider emerging technologies as powerful tools to enhance teaching, learning and student engagement, I learnt that I am still a learner, learning what I can achieve with these technologies (Veletsianos, 2008). I realised that learning is about the activities that lecturers do to achieve different levels of understanding (Biggs, 1997). I learnt to scale down my ideas to one specific tool that meet lecturers’ needs.
Honestly speaking, it has been a challenge to come with the prototype. The idea that I had from the beginning, changed completely. I had to move from one idea to another. The prototype has given me an opportunity to consider the objectives and goals of my prototype, available resources and technologies, my approach, activities that I planned and how they would meet the outcomes, how I was going to support the lecturers and the effectiveness of the activities.
I have learnt that I should plan properly and be clear about my aims of using technologies, and not be driven by my own needs to use technologies just because they are available.