Tuesday, May 3, 2011
Welcome fellow EIDT-6510 students!
This is an introductory (and temporary) blog post for EIDT-6510-2 Online Instructional Strategies.
Sunday, April 17, 2011
Reflection on EIDT-6130 Program Evaluation
At the conclusion of each course I am taking at Walden University
Eight weeks ago, I started another course in the Master’s program I am pursuing. At first impression, I figured I would be learning some new jargon for old concepts I have employed for years. I was mistaken. I was also intrigued.
The characteristic that separates evaluation from other research disciplines is the assignment of value, and the process of judging between what is mediocre and what is worthwhile. Evaluation also distinguishes itself from other forms of research through its function of initiating change, not only through criticism, but even more through its effect of changing the way we think about change. The process of evaluation creates a dynamic mindset of continual improvement. Improvements frequently happen before an evaluation is completed, just because questions were asked that prompt a response. If evaluation’s goal was the gathering of information, this effect of observation changing what is observed might be considered a problem, but change is the purpose of evaluation. (Fitzpatrick, Sanders, & Worthen, 2011)
The importance of evaluation as a means to effect change is also a liability. Significant ethical challenges exist because evaluations bring about change. Evaluations are conducted to meet the needs of stakeholders who may have varied and even conflicting interests in the outcome of the evaluation. A balanced view normally requires input from multiple perspectives. Evaluators need to be aware of all stakeholder perspectives in an evaluation, and they need to be aware of their own biases as they design an evaluation, and they need to disclose their biases when reporting their results. Since a balanced view requires multiple perspectives, multiple evaluations are often required. (Fitzpatrick, Sanders, & Worthen, 2011)
My experience planning an evaluation started with the preparation of a concept map (pictured above). A concept map is a diagram similar to a mind map, but a concept map illustrates relationships between ideas, and particularly, hierarchical relationships between ideas. The hierarchy of these relationships can be represented vertically or horizontally, but the map will generally show a progression of inputs, outputs, and outcomes. A concept map is useful to build an understanding of how an organization works, providing an intuitive view of various stakeholders and their interests in an organization. (Novak & Canas, Revised January 22)
The next step of the evaluation plan was a program analysis report that described the program, its history and stakeholders, contextual factors related to the stakeholders, and potential ethical challenges that would face an evaluation. A logic model was created to follow the activity of the program being evaluated. Like a concept map, a logic model considers the inputs, outputs, and outcomes of the program, and provides a framework for defining the kinds of questions an evaluation must answer to judge the effectiveness of a program. In the course of doing this assignment, I found an excellent template for designing a logic model which I downloaded from http://www.uwex.edu/ces/pdande/evaluation/docs/WorksheetExcel.xls
The wide variety of applications for evaluation have resulted in the development of a rich and varied set of evaluation models. Several categories of evaluation approaches were investigated, including expertise and consumer-oriented approaches, program-oriented approaches, decision-oriented approaches, and participant-oriented approaches. Strengths and weaknesses of each approach were reviewed, and a mixed approach was chosen that best matched the particular program characteristics and stakeholder interests with my own strengths and the limited resources available for conducting the evaluation. (Fitzpatrick, Sanders, & Worthen, 2011) The results of this investigation were documented in an evaluation model table. An evaluation criteria report more finely tuned the parameters of an evaluation plan and defined stakeholders and their interests, evaluation standards, and questions that the evaluation should seek to answer.
A data collection design and sampling strategy was developed as a PowerPoint presentation which was then recorded with the help of my daughter as an audio presentation in an interview format. The audio presentation discussed how data would be collected, specifically through interviews with various stakeholders, using census data to verify the validity of information obtained through interviews. The presentation discussed how the data collection design will address the evaluation questions. It discussed the need for random selection of stakeholder interviewees, and it discussed potential limitations to the data collection plan that will need attention.
A reporting strategy table was used to plan reporting strategies that will be appropriate to different kinds of stakeholders. In the reporting strategy, consideration was given to values and standards related to fair treatment of stakeholders, with a discussion of ethical policy.
I leave this class with an odd mixture of relief and regret: relief that an intense period of learning has come to a conclusion, (although personal circumstances may have influenced my perception of the intensity of the learning activities) and regret that the eight-week class format did not provide enough time to follow through to conduct the evaluations we planned.
Fitzpatrick, J. L. , Sanders, J. R. , & Worthen, B. R. ed .). Upper Saddle River , NJ
Novak, J. D. , & Canas, A. J.
Sunday, February 27, 2011
Reflection on EDUC-6145
 |
| (Kelly, 2009) |
Reflection on EDUC-6145 Project Management in Education and Training
At the conclusion of each course I am taking at Walden University
Rarely have I completed a course with the same level of enthusiasm for what I have learned. I have informally lead or managed many different kinds of projects over the years, but I have never felt I was good at it. In the past, I have written down my objectives, researched the topics involved, and have borrowed heavily from someone else’s plan, or better yet, found someone else to manage the project!
After taking this course, I’m already looking for ways to exploit and develop my new skills. I’m enthusiastic. I am especially enthusiastic about some of the online project management tools I have encountered over the past few weeks, and of those my greatest enthusiasm is for the tools that leverage online collaboration. My immediate plan is to start employing project management tools for the routine projects I manage all the time. Long-term, I think software tools can be over used. Sometimes a notebook and a pencil is the best tool for the job, but for now, I intend to leverage every project as an opportunity to develop my skills.
In the context of education and instructional design, I see potential for synergy in the similarities and differences between the disciplines of project management and instructional design. Design processes in any context involve iterative cycles of analysis, design, development, testing, and review of test results. Instructional design is no different. The design process involves testing and debugging, or in some contexts, trial and error. Design can be messy, and its results can be unpredictable. The messiness of the design process is where project management can assist. There is a danger that management objectives can stifle the design process, with quality suffering for the sake of a predictable timetable. However, planning of the development process with clearly defined objectives to prevent scope creep, can streamline the process without sacrificing quality. In practice, the structure provided by good management can benefit creativity by allowing the creator freedom to focus on design issues.
I have already used my project management knowledge to help a friend brainstorm an entrepreneurial project. When brain-storming, mind-mapping can be a great way to organize thoughts that seem to defy a standard outline. While there are some great mind-mapping software packages available, a pencil and paper is all it takes to begin a plan. Draw a circle in the center of a page, and put your main idea there. Then let the main idea sprout branches and roots, and see where the idea takes you!
Reference:
Thursday, February 10, 2011
Scope Creep
 |
| youngentrepreneur.com |
As a programmer, I encountered scope creep in the form of program changes that really should have been handled as new projects, but were instead added to current or recently completed projects. As with most projects, a programming project has defined users and a defined environment in which the software will be used. Assumptions are made about the equipment that will be operating the software. Assumptions are made about the kind of data that will need to be stored, and whether that data will be stored locally, or on a network.
One extreme example of a programming project that was negatively impacted by scope change began as an unofficial proof of concept program by an engineer who felt software could be written that would save time by assembling a PLC program from a list of standard operations. The initial proof-of-concept was written to program one test machine that was not actually connected to any equipment. The program allowed an engineer to select from a group of pre-defined sets of instructions to create a program to operate automated heavy machinery. Then it read back the program to demonstrate the instructions were sent properly. Next the program was altered so that it could store its instruction sequences to a program file on a laptop. When I thought the project was complete, I was suddenly confronted with several requests that pushed the capabilities of the original test design. Engineers wanted to be able to connect to a variety of machines, including completely different kinds of machines that stored information in different ways. New subroutines had to be written for each kind of machine that the program would interact with.
Some engineers wanted to use the program for troubleshooting a PLC that was off the network. They wanted to be able to connect to PLC’s directly or over the network. Then they wanted security protocols added so that all engineers could see the program code on the PLC, but only certain engineers could alter programs and change them. After network security protocols were added, they wanted to have the same capability whether the computer was connected via the network or when connected directly to the PLC.
Other departments began using the program. Suddenly slow operation of the program (that was designed to test a concept, not to actually run in a production environment) was identified as a cause of production delays. With many more people using the program, there were demands to override security by people who may or may not have understood the safety issues involved in using the program. Meanwhile other departments were demanding to use the program to read parameters from a spec database to standardize machine setups and eliminate the step of having engineers write the PLC programs.
Finally it became necessary to stop all changes to the old program. A new programming project was properly designed to replace the old program, which eliminated many of the conflicting uses of the old program, such as the ability to run on the network and from a laptop. The old program was retained for its ability to display existing programs, but it was no longer used to write and store new PLC programs.
In hindsight, the prototype program should never have been put into use in a production setting. At that point, a new project should have begun, involving all departments that would be using the program, to insure safety and to ensure performance demands for all possible users and environments for the program were considered and planned from the beginning.
References:
Portny, S. E., Mantel, S. J., Meredith, J. R., Shafer, S. M., Sutton, M. M., & Kramer, B. E. (2008). Tracking projects and maintaining control. In Project management: Planning, scheduling, and controlling projects. Hoboken , NJ John Wiley & Sons, Inc.
Thursday, February 3, 2011
Estimating Costs and Allocating Resources For Instructional Design Projects
Today’s blog assignment is to search for resources that would be useful in estimating the cost of instructional design projects. I began my search using Google Scholar with keywords like “Costing Instruction.” I found a number of useful resources that discussed the convergence of project management principles with instruction design principles, but nothing that directly discussed cost estimates. I broadened my search to include all web sites, but that search did not provide any useful resources either. Finally I searched specifically for the title of my blog assignment, “Estimating costs and allocating resources.” I got a couple of useful hits:
Estimating Costs and Allocating Resources In Instructional Design
December 2nd, 2010
http://www.ablsc.com/distance-education/estimating-costs-and-allocating-resources-in-instructional-design/
December 2nd, 2010
http://www.ablsc.com/distance-education/estimating-costs-and-allocating-resources-in-instructional-design/
This link could easily be another student’s post from a previous session of the same class I am taking. The author discusses two links providing useful information about costing instructional design projects. The first link is a paper describing a step-by-step process to cost development of an instructional web site. The second is a report describing a similar process to cost a software development project.
How to Estimate Training Time and Costs
May 18, 2010
http://ridgeviewmedia.com/blog/2010/05/how-to-estimate-trainingtime-and-costs/
May 18, 2010
http://ridgeviewmedia.com/blog/2010/05/how-to-estimate-trainingtime-and-costs/
This link is also a review of other websites that discuss costing of instructional design projects. Its resources include a paper published by the ASTD on the cost of developing one hour of instruction. The next resource is a forum discussion on the time required to develop a course. The last article discusses how to track project hours.
Then I tried searching the title of the last blog I found, “How to estimate training time and costs.” I found additional useful hits:
Simple Process To Estimate Times and Costs In A Web Project
byAntonio Lupetti
http://woork.blogspot.com/2009/02/simple-process-to-estimate-time-and.html
by
http://woork.blogspot.com/2009/02/simple-process-to-estimate-time-and.html
This link is a nicely illustrated guide to estimating the cost of a web project (not necessarily an instructional design project).
Estimating Costs and Time in Instructional Design
http://www.nwlink.com/~donclark/hrd/costs.html
http://www.nwlink.com/~donclark/hrd/costs.html
The last resource may be the most comprehensive resource I found, and it is geared specifically to instructional design projects. It discusses budgets, development costs, and development hours, with a focus on the resources required to develop one hour of training.
Thursday, January 20, 2011
Communicating Effectively
My assignment today is to blog about how I perceive a short message when it is delivered in an email versus over the phone and in person. The message states that a report is missing and that data from that report is needed. It requests an estimate about when the report can be ready, or more specifically, when the data that was to have been in the report could be provided. (“Communicating with stakeholders”)
Possibly because I have a minor learning disability, I did not perceive a difference in the message when it was delivered in different ways. I prefer email over the other forms of communication in this exercise, because I can get a general idea of the message at a glance, and I can re-read the details as needed for precision. I generally read everything this way, summarizing and outlining material on first glance, then going back for detail, the second time skimming over some paragraphs while focusing intently on others. I also prefer email because it presents the least interruption if I am in the midst of a project. Emails are often easier to handle quickly.
The voice mail example was my least favorite vehicle of delivery because I generally have to listen to voice mails repeatedly to catch all the details. Depending on the voice mail system, they can be tedious to use. Gathering the details from a voice mail can require replaying the message several times. I prefer a message I can interact with, like an email or a live person.
The last vehicle of delivery in the assignment was person-to-person. I have mixed feelings about person-to-person communication. One-on-one communication is effective for me, but I find formal meetings distracting and inefficient. If I need information from a meeting, I usually have to take additional time reviewing meeting notes and researching gaps in the notes to get the complete information I need.
References:
EDUC-6145-3 project management in education and training. (n.d.). Communicating with stakeholders: Blog assignment. Walden University
Thursday, January 13, 2011
Learning from a Project “Post-mortem”
As a means to learning “best practices” on future projects, I have been assigned to write a project post-mortem about a personal or professional project that was not successful. (“Defining the scope of the project: Blog assignment”)
I decided to write about an instructional design project that I have discussed previously in this blog. The project actually went quite well, but it definitely had its rough spots. The need for the project became apparent when I decided to try using Facebook as a tool to give ESL students experience using English. The attempt to use Facebook failed because the students lacked the basic skills to use a computer. I created a seminar to teach basic computer skills to adult ESL students, with a focus on using computers for social networking. (Lloyd , 2010)
In his eBook “The Project Management Minimalist: Just Enough PM to Rock Your Projects,” (Greer , 2010) Michael Greer suggests conducting a project “post mortem” involving asking of team members to find out their opinions of the success of the project, and brining respondents together for a “lessons learned” meeting. Greer suggests general questions that are appropriate for a post mortem, and then suggests “phase-specific questions.”
Below are my answers to some of Greer ’s questions related to the seminar I created for my students:
The single most frustrating aspect of the project was the amount of material I wanted to cover, versus the time that was allowed to cover that material. In the future, I would plan a series of at least three seminars to cover the material I attempted to cover in one evening. However, the primary objective of giving students the confidence to continue to learn to use a computer was clearly achieved. Students who had never used computers previously, expressed the intent to purchase computers and to continue learning to use them on their own. The “hands on” activities that were planned to give students immediate experience using a computer worked well. Students who attended the seminar left with new skills that will benefit them for life. The discussion portion of the seminar was difficult to evaluate because discussions happened in Spanish, and I do not speak Spanish. The interpreter participated in the discussions and concluded the discussions by interpreting summary comments.
The planning of needs and feasibility tended to err on the side of providing too much material for a one-session seminar, however the material that was planned was necessary in order to accomplish the goals of the seminar. The only change I could recommend for the seminar would be to allow for more sessions in order to cover all of the material. I would not change the seminar content.
Original estimates of the time required to accomplish project goals were not realistic. The focus on physical skills effectively overcame much of the potential communication difficulty of an English-speaking teacher working with Spanish-speaking students. Materials were effectively translated, and although the language barrier hindered classroom discussion, the social learning aspects of the plan worked effectively with the use of a translator.
The time estimate was off because the educational diversity of participating students, and the effect of that diversity on the flow of classroom activities was not anticipated. Testing and timing of classroom activities was done with students who understood instructions, and were already familiar with using a computer. None of the actual students had ever used a computer. Some of the adult students had less than three years of education in their own language, while other students had some college-level training. More focus on knowing the students would have enabled more accurate planning of time required to complete the seminar.
References:
EDUC-6145-3 project management in education and training. (n.d.). Defining the scope of the project: Blog assignment. Walden University
Subscribe to:
Posts (Atom)