The Importance Of Item Analysis

Two years ago, our headmistress called an urgent meeting where one of our district education supervisor was coming down to our school to give a talk on ways to improve the qualities of our students who were going to sit for their UPSR Exam. 

I still remember that one of the methods that can be used to improve their qualities after they done their monthly test is item analysis. But, being the greenhorn that I was, I pay less attention on the details and simply carry forward to "my" style of teaching.

It was not until this year that I "stumbled" the "item analysis" thing again during my Measurement and Assessment in Education class in OUM. This time, I had no chance to avoid it since one of my assignment's question needs me to conduct an item analysis on a given test to my sample students.

Since I put serious effort on undertaking my assignment, I really had to learn from the beginning about item analysis. At first, I find that item analysis is not worthwhile because most of my colleagues said that doing it is time-consuming and does not give a big impact on student learning as they are just numbers. But, when my tutor explains to me that item analysis is a crucial step to improve the quality of students in test-taking, it picked up my interest as I was almost ran out of idea how to improve my students in Science in the quickest and reliable way as part of their preparation in facing the UPSR exam.

After sourcing outside information, this item analysis that I have been doing for my student is actually known as the Toolkit Evaluation method. So, here I know that what I do is not anything new but proves to be a reliable method that can improve learning. (You can click here to get more info about Toolkit Evaluation).


By far, I've been using item analysis to get another two important data, which are difficulty index and discrimination index. Both of these data are actually the keystone to further improve the quality of question given to student, which supports better student learning through test-taking.


Difficulty index (of an item) is defined as the percentage of the overall students that can answer correctly the given question. The higher the number of index or nearer to 1, the easier the question is while the lower the number of index or nearer to 0, the harder the question is.

The formula used to calculate difficulty index is as below:

Difficulty index = number of total correct answer answered / number of total student answered

Here, the teacher can use the information to specify students' learning on learning topic that are related to the questions that has low index.

Discrimination index (on an item) is defined as the ability of a questionnaire to discriminate between high achieving students and low achieving students. A higher number of the index or nearer to 1 signifies that the question is reliable to be used to discriminate high and low achieving students while a lower score indicates otherwise.

To calculate the formula of discrimination index requires a few steps to be done. First, the total number of students sitting the test must be divided into three groups. The first 27% of the highest marks will be categorized as "high achievers". Then, 46% of the next highest marks students will be grouped into "medium achievers". The remaining students will be grouped into "low achievers".

After the division of groups is done, then we can use the formula below to calculate the discrimination index.

Discrimination Index = rU - rL / 0.5 (tU+tL)


rU = Number of high achievers who answers the given question correctly.
rL = Number of low achievers who answers the given question correctly


tU = Total number of high achieving students
tL = Total number of low achieving student


Because of these benefits, an item analysis is considered to be one of the important factor that can be used to improve students test taking abilities.

National Primary School Exam (UPSR) Science: Answering Technique For Section B

The National Primary School Exam (UPSR) is just around the corner. For most students, these are the "critical times" that they need to utilize properly in hope of getting a good result. But, as we all know, the national exams' contents is from a three-year study (Year 4, 5 and 6). It may prove to be a difficult situation to some students, especially those who has low memorization. However, memorization can be improved. According to this article, memorization can be improved and after I make a quick view of the contents, it uses techniques that are really simple and manageable, most importantly, fun for any youngsters. Still, memorization on Science facts only improves the ability the answer Section A of the National Primary School Exam for Science. For Section B though, it requires technical skills. There are three essential technical skills that are needed to answer Section B questions which are interpreting the questions, choosing the best and suitable answer script and time-managing skill.

Before I explain further, let me show you an example of a typical Section B question that being used for the National Primary School Exam.

National Primary School Exam for Science Section B (2010). Question No. 2

Interpreting the questions is actually the first phase and the most important technique of which student must do before engaging the sub-questions. Interpreting the question is actually requires you to accomplish three things.

1. Have a general idea of the science concept that is being "talk" in this question.

2. Identifying the variables required which are the manipulated variable (or independent variable), responding variable (or dependent variable) and fixed variable (or static variable).

3. Identifying the key questions asked based on the keyword used on the sub-questions.

Based on the example above, the science concept that is being tested here is measuring mass of an object by means of measuring the volume of water displaced when the object is immersed into a volume of liquid. also known as the "Archimedes Principle".

Clearly, the table in the question mentioned two kinds of objects, which are marbles and water. In Science, the element of a variable requires you to mention two things, the parameter and the object. According to Wikipedia.org, parameter is a term that is used to "identify a characteristic, a feature, a measurable factor that can help defining a particular system". For this question, the manipulated variable (or independent variable) is the "number" (parameter) of "marbles" (object) while the responding variable (or dependent variable) is the "volume" (parameter) of "water in measuring cylinder" (object). The fixed variable (or static variable) for this question would be the "size" (parameter) of "marbles" (object). 

Then, identifying the key question will be the next step as this will help you to determine what type of answer script is needed to answer these questions. The keyword, or we refer it directly the key question itself for 2a is "purpose", 2b(i) is "constant variable", 2b(ii) is "responding variable" and 2c is "conclusion". 

Now, we will proceed to the next step, that is choosing the best and suitable answer script. Based on the key questions that have been identified, this will ease us on providing the answer. The only restriction the students will face is to remember which script goes to which key question. Below, I have provided the list of key questions and its answer script. But, not all key question has an answer script because some key question requires some mathematical operation and logical reasoning based on science facts.

Key Question: Purpose
Answer Script: To investigate the relationship between the (manipulated variable) and the (responding variable).

Key Question: Relationship (or Hypotheses)
Answer Script: If the (manipulated variable)(trend pattern), then the (responding variable)(trend pattern)


Key Question: Conclusion 
Answer Script (1): When the (manipulated variable)(trend pattern), so will the (responding variable)(trend pattern)
Answer Script (2): The (trend pattern) the (manipulated variable), the (trend pattern) the (responding variable).
Answer Script (3): -make a general statement when one of the variables have no trend pattern-


Key Question: Observation
Basically, to answer this question, you need to make statement based on the result of the question given. It can be either in three types. 
Answer Script (1): The (responding variable)(trend pattern)
Answer Script (2): The (responding variable) of (object B) is much (adjective) than the (responding variable) of (object A) - when there are two objects of the same kind. 
For instance, the brightness of bulb in the parallel circuit is much brighter than the brightness of bulb in the serial circuit.


Key Question: Inference
Answer Script: (Observation) because (Science fact)
For example: The brightness of bulb in the parallel circuit is much brighter than the brightness of bulb in the serial circuit because the parallel circuit has less electrical resistance than the serial circuit.


Key Question: Variables (also Information)
Basically, just rewrite the variable requested.


Key Question: Trend Pattern (also known as changing trend and changing pole)
Answer Script: Increases / Decreases / Static or No changes 


Key Question: Predict
Basically, you need to do some logical reasoning or mathematical operation to answer the question.


Based on the question above, these are the answers for:
2(a) - To investigate the relationship between the number of marbles and the volume of water in the measuring cylinder.
2(b)i - Size of marbles
2(b)ii - Volume of water in the measuring cylinder
2(c) - When the number of marbles increases, so will the volume of water in the measuring cylinder increases.


Finally, time management is the key to success in answering all the questions in Section B. No matter how good you are in providing the answer, if you don't answer within the required time, no marks nor even extra time will be given. A good time division is required to get the best of Section B. For each main question, spend a maximum of 10 minutes per main question. My suggestion is give at least 4 to 5 minutes or less to do Step 1, that is interpreting the question and identifying the variables. Step 2 and 3 is a process that is done together. Spend about 3 - 4 minutes to answer the sub-question. Remember, try to allocate 40 minutes or so to answer all the questions in Section B.

Hope this guide will serve nicely to our youngsters who will be sitting their National Primary School Exam this September.

Cheers. ^^


Reference:

Archimedes Principle on Wikipedia.org. Available at http://en.wikipedia.org/wiki/Archimedes_Principle

Parameter on Wikipedia.org. Available at http://en.wikipedia.org/wiki/Parameter

Smith, M. & Robinson, L. (2012). How To Improve Your Memory. Help Guide Organization Portal. Available at http://www.helpguide.org/life/improving_memory.htm

Problem Based Learning in Science: The Advantages for Youngsters

In reality, no one has never faced a problem in life, even youngsters. Some youngsters are good at solving problems but some may not. Is it a human natural ability to solve problems? According to Wikipedia.org, problem solving is considered to be a "higher order thinking process which requires the modulation and control of routine or fundamental skills". According to Merriam-Webster students' dictionary, problem is defined as "something to be worked out or solved" (for arithmetic problems) or "something that is hard to understand, deal with or correct"(for social problems). Based on terms given, it is safe to say that problem solving is "a process to solve a certain condition using acquired fundamental skills". This would mean that, to succeed in problem solving, one own fundamental skills is heavily relied on and to be put on test, where the learning commence. So, how does problem based learning benefits youngsters in learning Science? In my point of view, problems based learning improves their understanding of a science concept better, decision making and most certainly, their problem solving skill.


Problem based learning is one of the Science learning method that helps youngsters to understand Science concept better. For instance, you can help youngster indulge to science concept by this example.

1. Show them a picture of a bouquet of wilted flowers. (acts as the problem)

Source: http://heathereverpresent.blogspot.com/2011/07/flower-power.html

2. Then, ask them these questions.
Q1: Why does the flowers wilt? (students' answers acts as an indicator to their Science concept understanding)
Q2: Is there any way we can help the flowers to look fresh again? (students' answers acts as an indicator how they used their science concept knowledge to solve problems)


Youngsters will give sorts of answers like lack of food, water, sunlight, space, etc and it is our responsibility to guide them by helping them explaining the concept if any misconception of facts are revealed. If none occurred, try to praise youngsters who give acceptable answers.

Decision making is one of the three benefits earned through problem-based learning. However, decision making or choices is somewhat difficult for youngsters. It is probably due to most the time, parents are the deciding factors to their child's needs, whether good in appearance, studies, social life etc that normally leaves youngsters "hard to make a good decision for themselves". According to a debate topic on DebateWise.org titled "Parents Should Let Teenagers Make Their Own Decisions", 14% of votes received are "NO". The fear of some parents not wanting their children to make a mistake creates an environment that are "error free" for youngster, which is non-beneficial for youngsters in their Science learning. As far as history goes, not even the greatest inventors of all time have build their invention without error. "Trial and error" is one of the techniques used in problem-based learning to solve problems.Through the trials and errors done, youngsters are perfecting their decision-making skill, with the hope that they will learn making better choices, thus minimizing bad choices.


Problem based learning in Science will help youngsters to be better problem solvers although problem-solving skills  may vary between youngsters. It is because different youngsters have different cognitive level. According to Gardner's Multiple Intelligence theory (1983), youngsters who have logical intelligence excel in areas that requires logical thinking, thus making them better problem solvers than youngsters that lacks logical intelligence. However, logical intelligence is trainable. By providing games that requires problem-solving such as chess, investigations or mystery solving, it will enhance their problem solving skill. The most significant benefit of being a good problem solver is it improves the ability of youngsters being independent. By applying problem-based learning in Science, it will automatically help youngsters to become independent learners.


Science is a fun subject to be explored and problem-based learning increases the excitement of it. Not only problem-based learning is applicable in Science learning, it also gives advantages to youngsters in being better individuals, in terms of their intelligence's quality.




Reference: 
Lane, C. (n.d.). Multiple Intelligence. The Distance Learning Technology Resource Guide. Available at http://www.tecweb.org/styles/gardner.html

Parents Should Let Teenagers Make Their Own Decisions on DebateWise.org. Available at http://debatewise.org/debates/1468-parents-should-let-teenagers-make-their-own-decisions

Problem Solving on Wikipedia.org. Available at http://en.wikipedia.org/wiki/Problem_solving