Reflection on Assessments in Online Environments

As this course on Assessments in Online Environments ends, I have been asked to reflect on the course and how it will affect my professional practice as a way to demonstrate my learning. Part of this assignment was to provide a narration of my reflection. Since blogger.com does not directly support embedded audio, I created an external link for the narration. Click the graphic to the left to play the narration.

Designing online assessments is part of a larger iterative process of designing online instruction. It begins with understanding the mission of the project: who are the learners, what are the learning objectives, and how will distance learners be able to demonstrate that they have incorporated the learning into their own day-to-day thinking. (Dick, Carey, & Carey, 2009)

In the past, I considered assessment my weakest area in instructional design. Now I feel I have a deeper understanding of the issues that separate shallow homework checks from the kinds of formative and summative assessments that allow students to take charge of their own learning, and instructors to focus on individual student needs. I understand that the validity of an assessment has to do with how well it connects with learning objectives. I understand the importance of ensuring “generalizability,” that the topics covered in an assessment provide an accurate picture of all learning, and not just the topics that were covered. I also understand how to achieve better generalizability through covering learning objectives from more than one perspective, and through using performance assessments when appropriate. (Oosterhof, Conrad, & Ely, 2008)

Concepts and Strategies

One of the benefits of this course was that it allowed me to look back at older learning models to realize each learning model provides essential tools for the instructional designer. In the first week of the course I commented that I had been so focused on social learning principles, that it was refreshing to consider that social learning requires each learner to have a foundation of concrete facts.

We discussed the difference between formative and summative assessments. Formative assessments are generally not graded, and can allow instructors and students to get quick non-threatening feedback to identify weak areas in student understanding and instruction that need strengthening. Sometimes formative assessments are used directly by students to make quick adjustments in their own learning. Sometimes instructors and instructional designers use formative assessments to identify problems with learning activities. (Oosterhof, Conrad, & Ely, 2008)

Later we discussed kinds of learning, and kinds of assessments that are appropriate to these kinds of learning. Declarative learning involves facts that can be memorized and repeated such as “Indianapolis is the capital of Indiana.” We discussed procedural or “rules” learning, which involves following a series of steps to complete a task. Learning how to make a pot of coffee would be an example of procedural learning. We discussed problem-solving which requires application of facts and rules to demonstrate learning. (Oosterhof, Conrad, & Ely, 2008)

In conjunction with different kinds of learning, we discussed various kinds of assessments that are appropriate to these kinds of learning, along with advantages and disadvantages of using these various kinds of assessments.

Fixed-answer quizzes, in which students select from a choice of predetermined answers, such as true-false questions, multiple choice, matching, and sequencing exercises all provide students with answers to be recognized or manipulated to demonstrate declarative or procedural learning. Fixed answer quizzes can provide students with instant feedback for formative assessments. They can also take less time to grade.
Disadvantages include the tendency for students to get no feedback on incorrect answers. Multiple-choice questions must have alternate choices that appear plausible to students who have not thoroughly mastered learning objectives, but which are obviously incorrect to students who have mastered the objectives. When multiple-choice questions are not well designed, they can mislead students who understand the material into selecting incorrect answers. They can also provide unintended clues to students who may use them to surmise correct answers when learning objectives have not been mastered. Designing effective multiple-choice questions can be time-consuming. (Oosterhof, Conrad, & Ely, 2008)
Short-answer  (one-word) questions have the advantage of not providing clues to students, but they have a disadvantage of requiring manual grading or review, because short-answer questions allow for the possibility of unanticipated correct answers. 
Essay and short-essay (phrase or sentences) questions are even more secure from the standpoint of accurately assessing student learning but require even more time on the part of instructors to grade. Unlike short-answer (one-word) questions, short-essay questions cannot be graded with an answer key. They require an instructor to read and subjectively evaluate each response. Checklists and rubrics can be useful to restore objectivity when grading short-essay questions. (Oosterhof, Conrad, & Ely, 2008)

Technology Toolbox

One especially useful tool from my “technology toolbox” that I used during the course was “iRubric” from http://www.rcampus.com/indexrubric.cfm. I have not fully explored the capabilities of the software, but not only does it guide the process of developing a rubric, including the ability to tie specific questions to learning objectives, but after the rubric is saved, it provides a utility to use the rubric to create individualized responses to students. 

Other tools I investigated during this course were Moodle (http://moodle.org/) and QuestionMark (http://www.questionmark.com/us/index.aspx). Moodle is a complete CMS system that is free, but requires installation on a server. I have installed Moodle on my personal website at http://1loyd.com/moodle. QuestionMark is a stand-alone assessment tool which was recommended in our course textbook as a software solution providing assistance with assessment design. (Oosterhof, Conrad, & Ely, 2008)
All of these tools provide automated assistance with grading. Many other tools were presented in the course “tech resources” section, including a number of online “gradebook” applications which I have yet to review, including: Engrade, MyGradeBook, SnapGrades, ThinkWave, TeacherEase, and TheGradeNetwork.

Insights About Scoring and Feedback

One aspect of online scoring and feedback from assessments that this course emphasized was the ability to pre-plan much of the student feedback that students need. Assessment software such as Moodle and QuestionMark has facilities to design and display question-specific and even answer-specific responses to guide students with instant formative feedback. Rubrics also provide a simple means to design student feedback by wording graded (point-based) mastery descriptions as suggestions to help students improve. (Oosterhof, Conrad, & Ely, 2008)
A significant insight I gained from this course was how to use scoring statistics to understand student responses, and in turn, to design more effective assessments. I learned how to recognize when apparent response patterns are significant, when they are not significant, and how to use significant results to identify problem questions that may need revision. (Suskie, 2009)

References:

Dick, W., Carey, L., & Carey, J. O. (2009). The systematic design of instruction (7th ed.). The systematic design of instruction: Pearson.

Oosterhof, A., Conrad, R.M., & Ely, D.P. (2008). Assessing Learners Online. Upper Saddle River, NJ: Pearson Education, Inc.

Suskie, L. (2009). Assessing student learning: A common sense guide (2nd ed.). San Francisco, CA: Jossey-Bass.

Inspiring online discussion

(Tapscott, 2009)

While some topics may inherently inspire interest, most topics require a controversial "twist" to make the topic interesting, and to cause readers to want to participate in the discussion. It is in finding that controversy or by adding something new from personal experience that makes discussion posts interesting and inspiring to read (Payne, 1985).

Consider how the topic of inspiring online discussion can be made interesting, and write a response to this post which expresses two ideas which can be compared and contrasted in a short response of two or three paragraphs. 

The scoring rubric may be downloaded from this link. Review the rubric before writing a response. 


References:

Tapscott, D. (2009, October 14). Student collaboration [Photo]. In wlibrary's photostream. flickr. Retrieved from http://www.flickr.com/photos/99107397@N00/4011844231/in/photostream 

Payne, L. V. (1985). The lively art of writing (5th ed.). Chicago, IL: Follett Educational Corporation.

REFLECTING ON ONLINE INSTRUCTIONAL PRACTICES

("MeisterLabs", 2010)

As this Online Instructional Practices course draws to a close, I have been asked to reflect on the course and how it will affect my professional practice as a way to demonstrate my learning.

How would you characterize your attitude toward best practices for online teaching and the role they will play with future design and teaching opportunities? How, if at all, has your attitude changed throughout this course?

Best practices for online learning are subtly different than those for traditional classrooms. Many principles, such as social and collaborative learning are the same, but they are accomplished in different ways than in a traditional classroom. The foundation of an effective online course is establishing a learning community between learners. (Boettcher & Conrad, 2010a) Learning activities that worked well in a classroom may or may not translate effectively in an online environment. Those activities that do not translate well need to be replaced. Sometimes the effectiveness of an activity may not be obvious. Trials of online training are a good way to determine the effectiveness of activities. Exit surveys are a good way to determine what material needs review and changes or replacement with other activities.

One best practice I have gained from taking this course has been understanding the value of exit activities such as this reflection assignment. Exit activities at the end of a course serve to reinforce the important principles that were learned, and provide one last opportunity for learners to incorporate principles into their personal conceptual framework. (Boettcher & Conrad, 2010a) Another best practice skill that I have improved as a result of taking this course has been the development of community-building activities. I have already found applications for ice-breaking skills I have learned in this course.

How do you see online instructional strategies blending with the instructional design process? How does it apply to the instructional design context?

Online instructional strategies such as asynchronous forums for discussion, logged chat sessions for synchronous collaboration, wikis for group projects, and videos and games to provide learning activities for visual and tactile learners are all tools in the arsenal of an instructional designer creating online instruction. No one strategy will meet all needs of various kinds of learners, but combinations of different types of activities can provide enough variety to enable learners to construct their own meaning and understanding of learning objectives. (Boettcher & Conrad, 2010b)

An instructional designer must choose from a variety of learning activities, those activities that will meet the needs of the greatest variety of learners, however the final responsibility in meeting the needs of individual learners rests with the learners, who must adapt those activities to their own learning styles. Successful learners take an active part in the construction of their own learning. Instructors serve as “mentors” (Boettcher & Conrad, 2010b, p. 23) in guiding learners to construct their own understanding.

Whether you are planning to design or facilitate online learning, what skills and competencies would you like to further develop?

It is clear to me from personal experience as a homeschooling parent, and from classroom activities in this course, that designing online assessments is currently my weakest area. Fortunately, my next course, “Assessments in Online Environments,” was designed to focus on strengthening assessment skills. I look forward to taking that class. Other skills I would like to improve, although they are currently my strongest skills, are my game development and general multimedia development skills.

Provide a specific example of how you will apply what you’ve learned to your current job or to a job you anticipate having in the future.

As a result of taking this course, I will be giving appropriate consideration to purposefully building learning communities out of my groups of online learners. I will be continuing that process throughout the course, even as I gently loosen the reins over the weeks, and eventually take on a less active roll with my classes, as they become more comfortable with taking charge of their own learning. (Boettcher & Conrad, 2010c) 

References:

Boettcher, J. V., & Conrad, R. (2010a). Ten best practices for teaching online. In The online teaching survival guide. San Francisco, CA: Jossey-Bass.

Boettcher, J. V., & Conrad, R. (2010b). Theoretical foundations. In The online teaching survival guide. San Francisco, CA: Jossey-Bass.

Boettcher, J. V., & Conrad, R. (2010c). Phase four: What's happening, themes, and toolls: Pruning, reflecting and wrapping up in the closing weeks. In The online teaching survival guide. San Francisco, CA: Jossey-Bass.

"MeisterLabs". (2010). Mindmeister. [Mind-mapping software]. München Germany: MeisterLabs. Retrieved from http://www.mindmeister.com

Plagiarism

Pirates (My daughter and friend dressed as pirates)

Plagiarism Detection and Prevention

From time to time, various classes in the course I am taking at Walden University require me to write blog entries as a way to demonstrate my ability to apply what I am learning to a practical issue. In my current class,” Online Instructional Strategies,” I have been asked to write about plagiarism detection and prevention.

As I considered the issue of plagiarism for this post, I reviewed the Walden University Student Handbook’s (2010) section on academic honesty. It defines plagiarism as the “use of intellectual material produced by another person without acknowledging its source.” (“Student handbook”, p. 244) The following descriptions were quoted from the handbook verbatim:

• Wholesale copying of passages from works of others into an assignment, paper, or discussion board posting, or thesis or dissertation without acknowledgment
• Using the views, opinions, or insights of another without acknowledgment
• Paraphrasing another person’s characteristic or original phraseology, metaphor, or other literary device without acknowledgment (“Student handbook”, 2010, p. 244)

The handbook adds a section on the use of one’s own previous work. The rationale for prohibiting the use of one’s own previous work is the need to insure that students are continuously building upon previous learning. As previous topics are reviewed in successive courses, it is natural that short references to previous work will be appropriate, but the University expects those references to be documented according to APA guidelines, as any other reference would be documented, and substantial use of previous work requires prior approval of the instructor of the course.

One of the perils facing online instruction is a perception that plagiarism is a greater problem for online students than for traditional classroom students, however actual levels of plagiarism appear to be similar for both modes of instruction (Palloff & Pratt). Rates of plagiarism have been difficult to quantify, due to lack of a standard method of detection. (Jocoy & DiBiase, 2006)

Plagiarism detection software has recently become a popular tool to quantify incidents of plagiarism and to restore academic integrity for students and institutions. Boettcher and Conrad (2010) list three websites that provide plagiarism detection for use by students and teachers:

• http://plagiarism.com/self.detect.htm
• http://turnitin.com/static/index.php
• http://www.grammarly.com/ (Formerly EssayRater)(Boettcher & Conrad, 2010, pp. 73, 74)

Another way to combat plagiarism is with the use of assessment methods that rely on application of learning rather than parroting of information. When students must apply what they have learned, the benefit of cheating is eliminated. Students want credit for their own work, and tend to have no motive for presenting the work of others as their own. (Palloff & Pratt)

Personally, I’m concerned that automated detection software can be too easily circumvented by resourceful, but lazy students. I’m tempted to illustrate what I mean by suggesting a few simple ways alterations of text could be automated using existing online tools, but it may be best to withhold those specific ideas. Any automated security software can be thwarted by other software. Encouraging students to “test” their own work for inadvertent plagiarism risks educating students about how to circumvent automated plagiarism detection. (Kaner & Fiedler, 2008)

I don’t find fault with instructors using software like TurnitIn as a tool to flag possible plagiarism, but software solutions are not perfect, and should be used with caution. I do not believe students have any need to use it at all. Software can return false positives when students use long phrases that are specific to a discipline. Automated methods exist which can reword direct quotations to prevent plagiarism software from recognizing them. Perhaps a more suitable method for detecting plagiarism might be to design software that recognizes patterns of grammar, and can flag documents that employ a patchwork of writing styles.

As a student, I have never been concerned about accidental plagiarism. In high school, I was taught to use index cards for research, writing a bibliographic heading on each card, and a paraphrase of the material I wanted to use. Having done my undergraduate work in the pre-computer era, I used the same system for research papers in college. I have always been proud of my ability to paraphrase and digest material, and I have long appreciated the usefulness of paraphrasing as a means to construct and verify my own understanding of the material I am studying. In fact, I frequently paraphrase and digest lectures as part of my note-taking process.

I still use this method in my research today. Although computer text files have replaced note cards for the most part, I still use note cards when I take lecture notes. I also rarely cut and paste anything. My writing process begins with an outline of assignment requirements. I write my own ideas about each assignment objective, and then I go back and find the citations that support what I have written. Sometimes I elaborate on what I have written by copying a section of material I had previously paraphrased. The result of my writing process is prose that I am confident is 100% original. Consequently, I have never bothered with anti-plagiarism tools to look for inadvertent plagiarism. I do not believe my method could allow plagiarism to happen.

However, there is one aspect of Walden’s definition of plagiarism that does concern me. While I don’t believe I have ever quoted myself verbatim, I often use the same words to express my ideas when I express those ideas again later. When I write about a topic I have written about before, I go back to that prior work to find citations. My writing is generally completed before I look for citations, and it is written in a style appropriate to the current context.

References:

Boettcher, J. V., & Conrad, R. (2010). Phase one: Tips for course beginnings. In The online teaching survival guide (pp. 73, 74). San Francisco, CA: Jossey-Bass.

(2010). Code of conduct. In Student handbook (p. 244). Minneapolis, MN: Walden University. Retrieved from http://catalog.waldenu.edu/content.php?catoid=23&navoid=2589

Jocoy, C., & DiBiase, D. (2006). Plagiarism by Adult Learners Online: A case study in detection and remediation. International Review of Research in Open amp; Distance Learning, 7(1), 1-15. Retrieved from http://content.ebscohost.com.ezp.waldenulibrary.org/ContentServer.asp?T=P&P=AN&K=21656952&S=R&D=ehh&EbscoContent=dGJyMNLe80Seprc4yOvsOLCmr0meqK9Sr6m4TbGWxWXS&ContentCustomer=dGJyMPGss0q1qK5IuePfgeyx44Dt6fIA

Kaner, C., & Fiedler, R.L. (2008). A Cautionary note on checking software engineering papers for Plagiarism. IEEE Transactions on Education, 51(2), 184-188. Retrieved from http://ieeexplore.ieee.org.ezp.waldenulibrary.org/stamp/stamp.jsp?tp=&arnumber=4455465&isnumber=4512114&tag=1

Palloff, R., and Pratt, K. (n.d.). Plagiarism and cheating [Video file]. Retrieved from http://sylvan.live.ecollege.com

The Impact of Technology and Multimedia

Impact of Technology and Multimedia
From time to time, various classes in the course I am taking at Walden University  require me to write blog entries as a way to demonstrate my ability to apply what I am learning to a practical issue. In my current class,” Online Instructional Strategies,” I have been asked to write about the impact of technology and multimedia to online learning. 


What impact does technology and multimedia have on online learning environments?

Technology impacts learning in a number of ways. It can focus attention or distract attention. It can bring out the best in people who normally may be too shy to contribute in a standard classroom, but it can also create social distance between learners, and stifle the creation of a learning community in a classroom. (Conrad & Donaldson, 2004) In the final analysis, online learning is just a vehicle delivering instruction. It has strengths and weaknesses in comparison with other vehicles. Perhaps its greatest strength is its ability to supplement traditional learning, and to provide more ways to reach more learners. (Neuhauser, 2002)

What are the most important considerations an online instructor should make before implementing technology?

The most important considerations in designing instruction are the needs of the learners, and the educational objectives to be accomplished. Meeting the needs of learners means establishing a social learning community that will give learners the confidence to work together, and providing clear direction that will enable learners to take charge of their own learning. (Boettcher & Conrad, 2010) It takes a lot of time and careful preparation, with adequate supplemental learning resources to meet the needs of every student. In a face-to-face classroom, my personal guide to success is to over-prepare, insuring there will be at least twice as many learning activities than I actually expect to use. An on-online classroom is no different. Learners need a variety of learning activities, and in the beginning, they need a lot of direction. Some learners will need help to start using online learning tools. Some just need to know the teacher is available in case they have difficulty. Another principle is to keep technology limited to what is familiar to both the learners and the instructor. Do not try to introduce several new technologies at once. Add one or two new technologies when it is second nature for the group to use the tools they already have. (Boettcher & Conrad)

What implications do usability and accessibility of technology tools have for online teaching?

I have a vision impairment. After two retinal reattachment surgeries, I have large blind spots in my peripheral vision, which the brain hides by creating an illusion of sight based on what it expects to see. The effect is a narrow range of vision in which reading is possible. Another side-effect of the surgery is that edges of the retina that were not reattached, float into the peripheral areas sometimes obstructing my view, while catching light from unusual angles, out of focus, and causing difficulty with contrast. The illustration above was designed to simulate how I see. It is a computer screen with edges blurred and darkened, and the center washed out with too much light, and with low text contrast. What is not clearly illustrated in the picture, is my own use of very large letters to overcome the problem, and the effect that large font sizes have on how text fails to wrap properly, or how navigation controls such as buttons and tabs can be completely hidden in badly designed web sites.

When designing online instruction, it is important to consider whether learners who are color-blind will be able to see screens that appear readable to most students. It is important to verify that that font sizes are proportional and that forms can be resized for vision-impaired learners. In many cases, verifying that instruction is accessible to learners with disabilities is required by law. Photographs must have alternate descriptive text for learners who use readers and cannot see pictures on the screen. Flash presentations that may not be accessible to blind users, need to have a transcript available. Just verifying that pictures and videos have alternate text available, or that fonts have relative sizes like “large,” and are not specified as a particular point size, may not always insure that a web page is readable to a person who is vision-impaired. Actual testing of learning materials using screen readers, resizing forms and choosing alternate color schemes is necessary to insure accessibility. (Burgstahler, 2006)

What technology tools are most appealing to you for online teaching as you move forward in your career in instructional design?

I am most impressed with the value of online assignment calendars, discussion forums, email, and online videos. Other collaboration tools such as wiki’s, blogs, video logs, and interactive games can be useful also. I also feel that learner use of spreadsheets and graphing tools can be important means for promoting self-discovery of relationships between data.

Synthesis

Clearly the most important principle I have learned since I started working on my degree in instructional design and technology, is the value of social learning, and how to create a learning community online, through careful direction by the instructor in the beginning, through “getting acquainted” activities at the beginning of a course, and through graded use of discussion forums throughout at course.

Much of the value of online learning is the flexibility it provides to learners to fit learning into their lifestyles. I expect to see more electronic books used in online courses to increase the portability of those courses. Text books and assigned articles should be available for direct download to a variety of electronic readers. In the future, I expect to see more discussion forums in online courses, with some of those forums becoming less formal, and more accessible to mobile devices. There will always be a need for some formality in courses, so that students are able to give proper credit to sources. Possibly future mobile “chat” software will provide a convenient utility for posting citations in chat messages.


Resources:

Boettcher, J. V., & Conrad, R. (2010). The online teaching survival guide. San Francisco, CA: Jossey-Bass.

Burgstahler, S. (2006). The development of accessibility indicators for distance learning programs. Research in Learning Technology, 14(1), 79-102. doi:10.1080/09687760500479753

Conrad, R., & Donaldson, J. A. (2004). Engaged learning in the online environment. In Engaging the online learner: Activties and resources for creative instruction. San Francisco, CA: John Wiley amp; Sons, Inc.

Neuhauser, C. (2002). Learning style and effectiveness of online and Face-to-Face instruction. The American Journal of Distance Education, 16(2), 99-113. Retrieved from http://www.personal.psu.edu/khk122/woty/LearnerCharacteristics/Neuhauser%202002.pdf

Setting Up An Online Learning Experience

From time to time, various classes in the course I am taking at Walden University require me to write blog entries as a way to demonstrate my ability to apply what I am learning to a practical issue. In my current class,” Online Instructional Strategies,” I have been asked to write about setting up an learning online environment.

As a student, when I begin a new course, I want to know what I will be learning, the kinds of activities that learning will involve, and the time I can expect to spend on those activities. I enter an online learning environment with questions like “how can I give myself the best advantage to stay ahead of course expectations?” I look for a course calendar, and I copy deadlines to my personal calendar, and set alarms so my cell phone will remind me of deadlines. I also look at specific activities with a focus on which activities demand an online connection, and which of those activities can be done off-line with some preparation. I download PDF files, convert them to Kindle format, and copy them to my Kindle reader. In an online course, fellow learners are a student’s most valuable resource. Taking an active roll in introducing myself to fellow classmates and making myself available to them as a resource is my next priority as I begin a new course.

In my previous post, I talked about online learning communities. Establishing an online learning environment is primarily about establishing a community of learners by acquainting each member with the tools of the online learning environment, and most importantly, by acquainting the members of the learning community with each other.

“The Online Teaching Survival Guide” (Boettcher & Conrad, 2010) suggests ten tips for setting up a successful online course. The first tip concerns course preparations, beginning with mundane considerations such as knowing who controls course resources and making sure those resources are in place. Beginning with the course syllabus, and following whatever procedural standards are required by the institution, the course designer maps out the evaluation plan and the course schedule, ensuring students will have reasonable time to complete learning activities. (Boettcher & Conrad)

Other tips concern how to make best use of time from the beginning of the course, how to design a syllabus that is appropriate for online learning, with special sections on communicating with fellow students online, understanding and avoiding plagiarism, and what to do when technical problems prevent timely posting of online assignments. (Boettcher & Conrad, 2010)


What is the significance of knowing the technology available to you?

Most schools offering online courses provide a course management system (CMS) which functions to standardize course structure and to streamline development. However, most CMSs do not provide tools for generating learning activities such as videos, games, or surveys. It is very important that students are familiarized with forum tools, email, and how to submit assignments using a CMS dropbox. (Boettcher & Conrad, 2010)


Why is it essential to communicate clear expectations to learners?

Especially in an online learning context, it is easy for students to be left with unclear understandings of classroom expectations. Misunderstandings of classroom expectations can happen as a result of not fully understanding the tools of the learning environment. Students who have less experience using email and online forums may need instructions in online etiquette to insure the learning community is supportive. Online tools make it easy to cut-and-paste materials from diverse sources. Students need clear direction regarding plagiarism, both to understand what it is, how to avoid inadvertent plagiarism, and how to make use of self-check tools.    (Boettcher & Conrad, 2010)


What additional considerations should the instructor take into account when setting up an online learning experience?

Don’t use technology for the sake of using technology, to be flashy, or just to prove you can. Select technological tools based on the learning objectives that are to be accomplished. As in most disciplines, the simplest approach that accomplishes the task is usually the best approach. Focus on facilitating each individual’s construction of their own mental model. Focus on facilitating social interaction that will enhance the number of personal connections students make with the instructional content. People learn socially. Use technology to bring people together, and avoid letting technology separate learners. (Boettcher & Conrad, 2010)


References:

Boettcher, J. V., & Conrad, R. (2010). Phase one: Tips for course beginnings. In The online teaching survival guide. San Francisco, CA: Jossey-Bass.

Online Learning Communities

From time to time, various classes in the course I am taking at Walden University require me to write blog entries as a way to demonstrate my ability to apply what I am learning to a practical issue. In my current class,” Online Instructional Strategies,” I have been asked to express my thoughts about the relationship between community building and effective online instruction.

A community is a group of people who come together to pursue a common goal. A community has members who have social relationships with one another. A community may be a neighborhood, a church, or a classroom. Recently the internet has brought about the existence of new kinds of communities that do not necessarily share a common location. Essential elements of a thriving online community are mutual trust and respect based on shared values or at least a shared code of conduct, some degree of personal social interaction, and in my opinion, a willingness to extend oneself to help fellow members of the community in whatever way may be needed. (Palloff & Pratt, 2011)

My daughter, using a cell phone camera, took the photo I have selected to illustrate a community. It was taken on the roof of my house last September. When I discovered I had a leak in my roof, I posted a plea to my friends on Facebook. A contractor who goes to my church replied that he would organize the work if I could find enough volunteers. He also helped me to select the best materials at the best price. A next-door neighbor, the husband of a coworker from my former job, a complete stranger who heard we needed help, and several Facebook friends from my church are among the people pictured. I was grateful and a bit overwhelmed by the reaction to my request for help. Together (my leg is visible in the gap under the arm of the unidentified man in the white sweatshirt) we stripped the shingles off the roof and loaded them onto a trailer to be hauled away, all in a single Friday. The next day, a somewhat different group of men began nailing shingles. With all the volunteer help, the job was finished within 36 hours. 

How do online learning communities significantly impact both student learning and satisfaction within online courses?

Online learning communities help each other to learn, by interactively sharing experiences, and filling in each other’s experiential “gaps,” enabling each member of the group to construct their own understanding in less time and with less effort than any of them could have learned on their own. Vygotsky (Conrad & Donaldson, 2004) described the difference between what individuals can learn on their own and what individuals can learn with the help of peers the “zone of proximal development.”

What are the essential elements of online community building?

The essential elements of online-learning community-building are the same as those of any community as mentioned above (mutual trust, shared values, social interaction, and helping one another). An online learning community potentially consists of students from many countries with potentially different cultural values, but the common learning goals and the structure of an online learning course provides common rules of conduct and mutual respect. In an online course, helping one another takes the form of discussion of ideas and mutual encouragement. Instructional designers must provide the necessary structure that will encourage students to collaborate, by creating appropriate rules of conduct and learning activities, including well-designed questions that encourage interactive creativity on the part of the community of learners. Such questions should focus not only on the immediate learning objectives, but should also encourage students to consider how they will use what they are learning in the future. (Palloff & Pratt, 2011)

How can online communities be sustained?

Although it may not always seem obvious to students, facilitators of online courses are constantly involved in various activities designed to support the online social learning dynamic. Especially at the beginning of a course, facilitators are continuously measuring levels of participation, and in recognizing when a student suddenly withdraws, possibly signaling a need to slow down and fill in gaps in the individual student’s conceptual background. Facilitators make comments that guide discussions and model profitable online behavior. Facilitators can strengthen the learning community as needed by scheduling live chat or conference calls over telephone or internet phone (Skype). Instructional designers can insure that the structure of a course supports the health of the online community. (Palloff & Pratt, 2011)

What is the relationship between community building and effective online instruction?

The online learning community is the primary means of providing effective online instruction. Books and videos can present information, but group interaction builds connections between course content and prior knowledge. Building and maintaining a learning community is more important to an online course than any other activity a facilitator undertakes. (Palloff & Pratt, 2011)

References:

Conrad, R., & Donaldson, J. A. (2004). Engaged learning in an online environment. 
        In Engaging the online learner: Activties and resources for creative instruction.
        San Francisco, CA: John Wiley & Sons, Inc.

Palloff, Rena, and Pratt, Keith. (2011). “Online Learning Communities” [Video file].
        Retrieved from http://sylvan.live.ecollege.com

Welcome fellow EIDT-6510 students!

This is an introductory (and temporary) blog post for EIDT-6510-2 Online Instructional Strategies.

Reflection on EIDT-6130 Program Evaluation

At the conclusion of each course I am taking at Walden University, I am required to write a brief reflection on the course. This reflection was written on the “Program Evaluation” class I just completed.

Eight weeks ago, I started another course in the Master’s program I am pursuing. At first impression, I figured I would be learning some new jargon for old concepts I have employed for years. I was mistaken. I was also intrigued. 

The characteristic that separates evaluation from other research disciplines is the assignment of value, and the process of judging between what is mediocre and what is worthwhile. Evaluation also distinguishes itself from other forms of research through its function of initiating change, not only through criticism, but even more through its effect of changing the way we think about change. The process of evaluation creates a dynamic mindset of continual improvement. Improvements frequently happen before an evaluation is completed, just because questions were asked that prompt a response. If evaluation’s goal was the gathering of information, this effect of observation changing what is observed might be considered a problem, but change is the purpose of evaluation. (Fitzpatrick, Sanders, & Worthen, 2011)

The importance of evaluation as a means to effect change is also a liability. Significant ethical challenges exist because evaluations bring about change. Evaluations are conducted to meet the needs of stakeholders who may have varied and even conflicting interests in the outcome of the evaluation. A balanced view normally requires input from multiple perspectives. Evaluators need to be aware of all stakeholder perspectives in an evaluation, and they need to be aware of their own biases as they design an evaluation, and they need to disclose their biases when reporting their results. Since a balanced view requires multiple perspectives, multiple evaluations are often required. (Fitzpatrick, Sanders, & Worthen, 2011)

My experience planning an evaluation started with the preparation of a concept map (pictured above). A concept map is a diagram similar to a mind map, but a concept map illustrates relationships between ideas, and particularly, hierarchical relationships between ideas. The hierarchy of these relationships can be represented vertically or horizontally, but the map will generally show a progression of inputs, outputs, and outcomes. A concept map is useful to build an understanding of how an organization works, providing an intuitive view of various stakeholders and their interests in an organization. (Novak & Canas, Revised January 22)

The next step of the evaluation plan was a program analysis report that described the program, its history and stakeholders, contextual factors related to the stakeholders, and potential ethical challenges that would face an evaluation. A logic model was created to follow the activity of the program being evaluated. Like a concept map, a logic model considers the inputs, outputs, and outcomes of the program, and provides a framework for defining the kinds of questions an evaluation must answer to judge the effectiveness of a program. In the course of doing this assignment, I found an excellent template for designing a logic model which I downloaded from http://www.uwex.edu/ces/pdande/evaluation/docs/WorksheetExcel.xls

The wide variety of applications for evaluation have resulted in the development of a rich and varied set of evaluation models. Several categories of evaluation approaches were investigated, including expertise and consumer-oriented approaches, program-oriented approaches, decision-oriented approaches, and participant-oriented approaches. Strengths and weaknesses of each approach were reviewed, and a mixed approach was chosen that best matched the particular program characteristics and stakeholder interests with my own strengths and the limited resources available for conducting the evaluation. (Fitzpatrick, Sanders, & Worthen, 2011) The results of this investigation were documented in an evaluation model table. An evaluation criteria report more finely tuned the parameters of an evaluation plan and defined stakeholders and their interests, evaluation standards, and questions that the evaluation should seek to answer.

A data collection design and sampling strategy was developed as a PowerPoint presentation which was then recorded with the help of my daughter as an audio presentation in an interview format. The audio presentation discussed how data would be collected, specifically through interviews with various stakeholders, using census data to verify the validity of information obtained through interviews. The presentation discussed how the data collection design will address the evaluation questions. It discussed the need for random selection of stakeholder interviewees, and it discussed potential limitations to the data collection plan that will need attention.

A reporting strategy table was used to plan reporting strategies that will be appropriate to different kinds of stakeholders. In the reporting strategy, consideration was given to values and standards related to fair treatment of stakeholders, with a discussion of ethical policy.

I leave this class with an odd mixture of relief and regret: relief that an intense period of learning has come to a conclusion, (although personal circumstances may have influenced my perception of the intensity of the learning activities) and regret that the eight-week class format did not provide enough time to follow through to conduct the evaluations we planned.

References:

Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2011). Program evaluation: Alternative approaches and practical guidelines (4th ed.). Upper Saddle River, NJ: Pearson Education, Inc.

Novak, J. D., & Canas, A. J. (Revised January 22, 2008). The theory underlying concept maps and how to constuct then [Electronic mailing list message] [Regular]. Technical report IHMC Cmap Tools 2006-01 Rev 01-2008, Florida Institute for Human and Machine Cognition, 2008. Retrieved from http://cmap.ihmc.us/Publications/ResearchPapers/TheoryUnderlyingConceptMaps.pdf

Reflection on EDUC-6145

(Kelly, 2009)

Reflection on EDUC-6145 Project Management in Education and Training

At the conclusion of each course I am taking at Walden University, I am required to write a brief reflection on the course. This is the reflection I  wrote on the ”Project Management”class I just completed.

Rarely have I completed a course with the same level of enthusiasm for what I have learned. I have informally lead or managed many different kinds of projects over the years, but I have never felt I was good at it. In the past, I have written down my objectives, researched the topics involved, and have borrowed heavily from someone else’s plan, or better yet, found someone else to manage the project!

After taking this course, I’m already looking for ways to exploit and develop my new skills. I’m enthusiastic. I am especially enthusiastic about some of the online project management tools I have encountered over the past few weeks, and of those my greatest enthusiasm is for the tools that leverage online collaboration. My immediate plan is to start employing project management tools for the routine projects I manage all the time. Long-term, I think software tools can be over used. Sometimes a notebook and a pencil is the best tool for the job, but for now, I intend to leverage every project as an opportunity to develop my skills.

In the context of education and instructional design, I see potential for synergy in the similarities and differences between the disciplines of project management and instructional design. Design processes in any context involve iterative cycles of analysis, design, development, testing, and review of test results. Instructional design is no different. The design process involves testing and debugging, or in some contexts, trial and error. Design can be messy, and its results can be unpredictable. The messiness of the design process is where project management can assist. There is a danger that management objectives can stifle the design process, with quality suffering for the sake of a predictable timetable. However, planning of the development process with clearly defined objectives to prevent scope creep, can streamline the process without sacrificing quality. In practice, the structure provided by good management can benefit creativity by allowing the creator freedom to focus on design issues.

I have already used my project management knowledge to help a friend brainstorm an entrepreneurial project. When brain-storming, mind-mapping can be a great way to organize thoughts that seem to defy a standard outline. While there are some great mind-mapping software packages available, a pencil and paper is all it takes to begin a plan. Draw a circle in the center of a page, and put your main idea there. Then let the main idea sprout branches and roots, and see where the idea takes you!

Reference:

Kelly, Will. (2009, December 14). Project Management Tools: Beyond Gantt Charts [Web log post]. Retrieved from http://gigaom.com/collaboration/project-management-tools-beyond-gantt-charts-2/

Scope Creep

youngentrepreneur.com

Scope creep refers to changes to targeted objectives after a project has begun. In my experience it generally results from failure to clearly define objectives initially, although it has also happened after initial prototyping when others begin to see potential applications for the project that are outside the initially planned context. (Portny, et al., 2008)

As a programmer, I encountered scope creep in the form of program changes that really should have been handled as new projects, but were instead added to current or recently completed projects. As with most projects, a programming project has defined users and a defined environment in which the software will be used. Assumptions are made about the equipment that will be operating the software. Assumptions are made about the kind of data that will need to be stored, and whether that data will be stored locally, or on a network.

One extreme example of a programming project that was negatively impacted by scope change began as an unofficial proof of concept program by an engineer who felt software could be written that would save time by assembling a PLC program from a list of standard operations. The initial proof-of-concept was written to program one test machine that was not actually connected to any equipment. The program allowed an engineer to select from a group of pre-defined sets of instructions to create a program to operate automated heavy machinery. Then it read back the program to demonstrate the instructions were sent properly. Next the program was altered so that it could store its instruction sequences to a program file on a laptop. When I thought the project was complete, I was suddenly confronted with several requests that pushed the capabilities of the original test design. Engineers wanted to be able to connect to a variety of machines, including completely different kinds of machines that stored information in different ways. New subroutines had to be written for each kind of machine that the program would interact with.

Some engineers wanted to use the program for troubleshooting a PLC that was off the network. They wanted to be able to connect to PLC’s directly or over the network. Then they wanted security protocols added so that all engineers could see the program code on the PLC, but only certain engineers could alter programs and change them. After network security protocols were added, they wanted to have the same capability whether the computer was connected via the network or when connected directly to the PLC.

Other departments began using the program. Suddenly slow operation of the program (that was designed to test a concept, not to actually run in a production environment) was identified as a cause of production delays. With many more people using the program, there were demands to override security by people who may or may not have understood the safety issues involved in using the program. Meanwhile other departments were demanding to use the program to read parameters from a spec database to standardize machine setups and eliminate the step of having engineers write the PLC programs.

Finally it became necessary to stop all changes to the old program. A new programming project was properly designed to replace the old program, which eliminated many of the conflicting uses of the old program, such as the ability to run on the network and from a laptop. The old program was retained for its ability to display existing programs, but it was no longer used to write and store new PLC programs.

In hindsight, the prototype program should never have been put into use in a production setting. At that point, a new project should have begun, involving all departments that would be using the program, to insure safety and to ensure performance demands for all possible users and environments for the program were considered and planned from the beginning.

References:

Portny, S. E., Mantel, S. J., Meredith, J. R., Shafer, S. M., Sutton, M. M., & Kramer, B. E. (2008). Tracking projects and maintaining control. In Project management: Planning, scheduling, and controlling projects. Hoboken, NJ: John Wiley & Sons, Inc.

Estimating Costs and Allocating Resources For Instructional Design Projects

Today’s blog assignment is to search for resources that would be useful in estimating the cost of instructional design projects. I began my search using Google Scholar with keywords like “Costing Instruction.” I found a number of useful resources that discussed the convergence of project management principles with instruction design principles, but nothing that directly discussed cost estimates. I broadened my search to include all web sites, but that search did not provide any useful resources either. Finally I searched specifically for the title of my blog assignment, “Estimating costs and allocating resources.” I got a couple of useful hits:

Estimating Costs and Allocating Resources In Instructional Design
December 2nd, 2010
http://www.ablsc.com/distance-education/estimating-costs-and-allocating-resources-in-instructional-design/

This link could easily be another student’s post from a previous session of the same class I am taking. The author discusses two links providing useful information about costing instructional design projects. The first link is a paper describing a step-by-step process to cost development of an instructional web site. The second is a report describing a similar process to cost a software development project.

How to Estimate Training Time and Costs
Posted on May 18, 2010 by Jenise
http://ridgeviewmedia.com/blog/2010/05/how-to-estimate-trainingtime-and-costs/

This link is also a review of other websites that discuss costing of instructional design projects. Its resources include a paper published by the ASTD on the cost of developing one hour of instruction. The next resource is a forum discussion on the time required to develop a course. The last article discusses how to track project hours. 

Then I tried searching the title of the last blog I found, “How to estimate training time and costs.” I found additional useful hits:

Simple Process To Estimate Times and Costs In A Web Project
by Antonio Lupetti
http://woork.blogspot.com/2009/02/simple-process-to-estimate-time-and.html

This link is a nicely illustrated guide to estimating the cost of a web project (not necessarily an instructional design project).

Estimating Costs and Time in Instructional Design
http://www.nwlink.com/~donclark/hrd/costs.html

The last resource may be the most comprehensive resource I found, and it is geared specifically to instructional design projects. It discusses budgets, development costs, and development hours, with a focus on the resources required to develop one hour of training.