Item evaluation of the English test of the Myanmar university entrance examination
English test item types and scores
The current version of the English test is composed of nine sections that measure reading and writing skills. The test comprises approximately 50% reading and 50% writing. A score of 40 out of 100 is the minimum passing score and 75 out of 100 is the distinction score.
Since the exam is taken at the same time in all schools throughout the country, it can be said to be a reliable test. As the exam is intended to test only reading and writing skills in accordance with the objectives of the course, it has apparent face validity since validity is defined as “the extent to which a test measures what it is supposed to measure” Brown, (2000). The current version of the English tests contains nine sections intended to measure reading, writing, and grammar skills. The language skills and areas upon which teachers and students focus during the test preparation process consist of reading of the textbook, writing, and grammar skills. Accordingly, the test is presumed to have high face validity because the content of the test appears relevant and appropriate. At the same time, one should also consider how exam questions evaluate a student’s subject-matter competency. Hughes (2003) explains, “in order to judge whether or not a test has content validity, we need a specification of the skills or structures, etc. that it is meant to cover” (p. 26).
Reading skill assessment sections of the English test
The test section on reading constitutes half of the total score. Questions intended for assessing reading skills consisted of reading comprehension, cloze-type questions, fill the blanks, and Q & A. In an analysis of test questions, it was found that all the comprehension passages are directly taken from the textbook. This encourages memorization of the textbook and this leads to questionable washback in that students may neglect the skills needed for reading a wider range of material. Students have already read and studied repeatedly the passages in the textbook during the preparation process as well as practicing the same passages from previous test questions. They thus become very familiar with the passages in the test and, assuming minimal preparation, can answer quite easily.
The following examples (1-9) are included verbatim from the English language test section of the university entrance examination for Yangon candidates in 2009 (Myanmar Board of Examinations, 2009).
Example 1: Reading Comprehension I & V
I. Read the passage
Dengue fever is a viral disease carried by the Aedes mosquitoes. The disease occurs mainly in tropical and subtropical regions and is most common during the rainy season in areas infested with the infected mosquitoes. According to the WHO, 2.5 billion people are at risk from the four major types of dengue virus, which is transmitted to people from infected mosquitoes. Each year there are tens of millions of cases.
Symptoms of dengue fever generally start 5 to 6 days after being bitten by the infected mosquito and include fever, painful headaches, eye, joint, and muscle pain, and rash. The rash usually begins on the arms or legs about 3 to 4 days after the fever starts. These symptoms resolve completely within 1 to 2 weeks. Dengue hemorrhagic fever is a severe, potentially fatal infection that occurs when someone with immunity to one type of dengue virus is infected by a different type. Its symptoms include loss of appetite, vomiting, intense abdominal pain, shock and bleeding from the nose or under the skin. The occurrence of dengue hemorrhagic fever may be associated with those either experiencing their second infection, being immuno-suppressed, or being around the age of 15.
(A) Write the correct word or words to complete each sentence. (5 marks)
Aedes mosquitoes -------------- dengue fever, a viral disease.
There are -------------- types of dengue virus.
Infected mosquitoes -------------- dengue virus to people.
After being bitten by the infected mosquito, ------------- of dengue fever usually start within 5 or 6 days.
One of the symptoms of ---------------- is the loss of appetite.
(B) Answer each question in one sentence.(5 marks)
Where does dengue fever mainly occur?
What are the symptoms of dengue fever?
When does the rash usually begin?
What is dengue hemorrhagic fever?
What are the symptoms of dengue hemorrhagic fever?
V. Read the passage
Robots are growing in complexity and their use in industry is becoming more widespread. Robots are mainly used in working with automatic machines and equipments in mass production industries, where the same tasks must be repeated. Car production is the primary example of the employment of large and complex robots. They are used in the process for the painting, welding and assembly of the cars.
Robots are also useful in environments which are unpleasant or dangerous for humans to work in, for example, bomb disposal, work in space or underwater, in mining and for the cleaning of toxic waste. Robots are also used for patrolling these toxic areas.
Nowadays, domestic robots that perform simple tasks such as vacuum cleaning and grass cutting are now available. Other domestic robots are being produced with the aim of providing companionship or play partners to people.
Answer these questions. Give complete answers. (10 marks)
It is doubtful such questions can measure reading ability accurately since students often do not read the passages. Two high-school teachers interviewed were adamant that if students mastered the entire textbook and old test questions, they need not be bothered by whether or not they possess basic reading skills. That said, students must understand changes of word forms in fill in the blank exercises as well as know how to answer different wh-type questions. There are no differences between the questions and answers of the reading comprehension tasks that appear in the textbook and on the test itself. It can be seen that one who has not read the textbook as many times as others can still get the right answers if he or she knows how to respond to wh-questions and comprehend the given passages. A better approach would be to present an unfamiliar passage with uniform standards to measure student reading comprehension.
Part III of the reading section requires students to fill gaps with correct words or phrases. Part IV asks students to give short answers and paraphrase. Both of these parts test what one has read and memorized from the textbook (called a recall type question), again becoming a test of memory more than of language proficiency. Students must memorize all poems prescribed in textbook to perform well in part III (see the following example 2). As apparent in part III, tests are geared toward rote memorization, not critical analysis; indeed, memorization is often what differentiates high scores from low ones.
Where are robots mainly used?
In which kind of processes are the large and complex robots used in car production?
Give some examples of unpleasant or dangerous works for humans.
What simple tasks can domestic robots perform?
Besides doing simple tasks, with what aim are domestic robots being produced?
Example 2: Example of Part III
III. Write the appropriate words or groups of words to complete the sentences and lines of verse. (10 marks)
The sections are poor measures of a student’s reading comprehension. Students are able to fill in the gaps with correct words simply from memory. Concerning washback, because the sentences and questions used on the test are the same as ones in the text, it stands to reason that students memorize passages instead of increasing their reading abilities.
Looking at the cloze type questions, the passage is exactly the same as that from the textbook. Students thus learn the passages by heart or read them repeatedly as shown in the following example.
The poor astrologer was -------------- out of his wits.
After glassblowing was invented, the --------------- came into use.
Advertising assists a ------------- of goods at reasonable prices.
The Egyptian calendar worked like a ------------------ that runs slow.
Helen ------------- from Radcliffe college in 1904.
A railroad bridge, Where freight trains -----------:
Whose trees in summer --------------- him shade, In winter, fire.
No time to see, in broad daylight, Streams full of stars, like --------------- at night:
And even as I -------------, It split itself in two.
Stars fall and --------------- In keen November,
Example 3: Example of Cloze Type Section II
II. Fill in each numbered blank with a word from the list given.(10 marks)
There are lots of jokes about snoring, --- (1) --- it isn’t really funny. People snore because --- (2) --- have trouble breathing while they are asleep. --- (3) --- snorers have a condition called sleep apnea. --- (4) --- stop breathing up to thirty or forty --- (5) --- an hour because the throat muscles relax --- (6) --- much and block the airway.
Most people --- (7) --- from 7½ to 8½ hours of sleep --- (8) --- night, but this varies with individuals. Babies --- (9) --- eighteen hours, and old people need less --- (10) --- than younger people. If someone continually sleeps --- (11) --- than normal for no apparent reason, there --- (12) --- be something physically or psychologically wrong.
You --- (13) --- save hours of sleep the way you --- (14) --- money in the bank. If you have --- (15) --- 5 hours of sleep for three nights, --- (16) --- don’t need to sleep an extra 9 --- (17) --- on the weekend. And it doesn’t do --- (18) --- good to sleep extra hours ahead of --- (19) --- when you know you will have to --- (20) --- up late.
Cloze-type is an integrated skill test, where particular language skills are not tested, but rather a mix of grammar skills, syntax and rhetoric, and critical analysis. All interviewees responded that the more they read the textbook, the easier they were able to complete the cloze-type section of the test. They produced the correct answers very easily in a short amount of time because they had already seen and read the textbook. The cloze-type section in the test itself, therefore, is an inadequate measure of student reading and writing ability. It degrades construct validity because “the greater a test’s content validity, the more likely it is to be an accurate measure of what it is supposed to measure, i.e. to have construct validity” (Hughes, 2003, p. 27). If the passages had not been previously studied, students would have to consider the content words, blank occurrences, word classes/ forms, and sentence meaning – which can effectively measure students’ language ability. In this way, the cloze-type section would succeed in measuring the examinees’ language ability. As Hughes (Ibid., cited in Kuroki, 1996) states, “… to respond to them [the items] successfully, more than grammatical ability is needed…”
In addition, part IV, which is question & answer, asks about items from the textbook and assesses what students have learnt as we can see in the following.
Example 4: Example of Part IV
IV. Give complete answers to these questions. (10 marks)
At what temperature are meat and fish kept in refrigerated self-service cabinets? (OR)
What is the capital of Cambodia and where is the Silver pagoda located?
When and how can stars be seen on the moon? (OR)
Say something about these lines (from your prescribed text)
Who said these words and to whom were they said? What do they mean?
What do these lines mean?
I shot an arrow into the air ______ It fell to earth, I knew not where;
Again if the students have read the text many times, they can answer the questions. Unsurprisingly, the teachers train their students to read the textbook a lot and review the questions from old test papers. The students state that this section is one of the more challenging sections of the test as they must remember the questions from the textbook and try to memorize the passages as much as they can during the preparation process. Grammatical knowledge is also required to respond to the questions.
Two teachers interviewed mentioned that most of the questions in this section ask distinct facts that students can easily recall and answer. Other teachers suggested that other areas of English should be studied and added to the test (e.g., article usage). The researcher agrees with the second statement because ability to read in a language is much more than mere memorizing. Although this kind of question is intended to assess the appreciation and paraphrasing skills of students, experienced teachers have found that very few students can perform paraphrasing with their own ideas and sentences. Moreover, few teachers focus on ways of approaching poetry both literally and metaphorically. Like other subjects, teachers train students to approach poetry through the lens of memorization. Clearly, students learn the paraphrased sentences of each poem given by their teachers in order to attain full marks. “Too often the content of tests is determined by what is easy to test rather than what is important to test” (Hughes, 2003, p.27).
What is the title of the unit from which the following sentence is taken? What does the underlined word refer to?
Writing skill assessment section of the english test
There are five sections in the test to measure writing ability. Most sections focus on the correct use of punctuation, editing ability, guided writing (giving prompts), letter writing, and writing cogent arguments. Turning to punctuation section VI (see Example 5 supra), we may see that based on the items used in the test, validity is dubious as a number of items used in the test are directly taken from the textbook that students have repeatedly seen and studied before. Frequently asked types of sentences are direct speech, giving orders, and making requests. Some brainstorming is required and students must apply their knowledge about punctuation which can improve their writing skill.
Example 5: Example of Punctuation Section VI
VI. Punctuate the following. (5 marks)
Alternatively, the Sentence Transformation Section (VII A) of the test can be regarded as valid because it tests not only the intended grammar skills presented in the course book but also fulfills the purpose of measuring grammar expertise (See Example 6). One commentator notes, “a good test of grammar may be valid for measuring the grammatical ability of the examinees but not for measuring other kinds of abilities” (Farhady, 1986).
Let us now look at how the teachers instruct and their students learn to improve grammar skills. The students interviewed are confident from grammar study in the past and since they have already mastered the grammar patterns, often with the help of the mnemonics devices. Surprisingly, some respondents stated that although they know how to give the right answers to grammar test sections, they cannot apply that knowledge in their own writing and analysis. They do not know the meaning of grammar patterns and the usage. From the interview results, it was found that students can do the grammar exercises very well simply by studying questions from previous exam papers. Teaching grammar reveals a disparity between those students who were good at English and those who were poor at English. T3 with 20 years English teaching experience also states:
For those who are good at English, we teach not only patterns/forms but also meaning whereas we give grammar rules using formula to those who are poor at English and train them a lot to remember those, then explain sentence changes giving examples, so they can get the right answer just following the formula but not knowing the meaning. But we at least try to get them to recognize the subject-verb agreement and exceptions that are not covered by formula. (T3)
This demonstrates that students who are weak in grammar have to learn the essential patterns as an intensive preparation while for those who aim for distinction must be proficient in grammar patterns. Though the students are good at grammar, they cannot apply these patterns in their writing tasks. Sometimes students are taught using syntax templates which allow students to appear adept at grammar for testing. Teachers should teach students to apply what they have learned by transforming their exercises into hands-on writing projects. Otherwise, these skills will be useless. Negative washback occurs through the exam-centric teaching system, where most of the teachers teach grammar and the students have to learn as it is. Two teachers with over 20 years teaching experience then explain:
Effective strategies - that doesn’t mean grammar strategies - focus on bringing more marks to the test. So students can get the right answer just following the formula but not knowing the meaning. (T7 & T3)
Most students regard grammar and writing as different subjects. They do not appreciate the overlap between grammatical constructions and skilled writing.
the astrologer said hey ive got all the tender bael leaves that should go well with our meal
i said when you understand that theres a word for everything the world will be yours
dont play computer games all the time nyi nyi said ko ko
the girl exclaimed what a beautiful doll it is
myanmar known as the golden land is famous for its glittering pagodas
Example 6: Example of Sentence Transformation Section VII A
The thief entered the house.
“Don’t make the same mistake again,” the teacher said to Kyaw Kyaw.
Health is more important than wealth.
Han Zaw Tun is so obedient that every teacher likes him.
Although Mu Mu is young, she is very intelligent
Yangon is the biggest city in our country.
I’ve been to Thailand. I’ve been to Singapore.
All the students from our school are polite and well-behaved.
As soon as they set out, it began to rain.
No sooner -----------------------------------------.
Concerning Section VII (B) of the test which involves constructing a paragraph using given prompts (called Guided Writing), the optional paragraphs come directly from the textbook, but the sentence structures are different.
(A). Finish each sentence in such a way that it means exactly the same as the sentence that is given. (10 marks)
The maid arranged the flowers in the vase beautifully.
Example 7: Example of Guided Writing Section VII (B)
(B) Using the prompts given, write a very short paragraph on ONE of the following: (10 marks)
Bangkok – capital – Thailand – it – regarded as – shopper’s paradise
one – enjoy the twin pleasures – Thai cuisine – classical dance – some restaurants
dancers use – movements – interpret – stories – the “Ramakien,” – Thai version – Ramayana
culture lovers – watch dance – drama – such venues – Thailand’s National Theatre, -- Thai Cultural Centre – Monthienthong Theatre
Pattaya – Phuket – the famous beach resorts – Thailand
they offer – wide range – activities – scuba diving – windsurfing – golf
Thai boxing – become popular – recent years
It – a noisy – exciting game which – accompanied – orchestral music – well worth experiencing
short, plump woman of – fair skin, brownish hair, -- feminine – prim
dreamt – having diamond earrings – dream came true
husband gave – diamond earrings
times – hard – had – pawn – diamond earrings
sewed rosettes – silk garters – penny piece – night
paid – interest – on pawn ticket every year
one summer – couldn’t afford – pay – interest anymore – surrendered
son became – writer – book – he wrote became – best seller in 16 countries
For material not identical to that found in textbooks, students must compare the passage with the story they had learnt in the textbook and rewrite it using the correct grammar and usage (especially linkers and correct tense). In preparation, it is common for students to read all the units in the prescribed textbook thoroughly and study previous test questions so they can extract the themes of the stories of the given paragraphs in the test; if not, it is challenging to rebuild the paragraphs coherently. The Guided writing section can be said to measure their ability to synthesize grammar, rhetoric, and idiomatic writing.
Regarding writing skills, some students state that the Essay and Letter Writing Sections are the most difficult test sections. The Letter Writing Section (e.g., Section VIII) can mirror “real life” writing (see Example 8). The questions include information about the context such as the person to whom the letter is supposed to be written and the purpose of the letter. During the test preparation process, students review writing addresses, dates, salutations, and ways of ending a letter in the correct format. Students also review how to include general sentences in the opening and closing paragraphs of the letter and include their own sentences in the body of text. This “hands-on” approach benefits students. However, results vary depending on the student’s region, economic clout, and overall culture. The variations that exist among different states and divisions lessen the reliability of the test
Example 8: Example of Letter Writing Section of the Test
You are Tun Tun. Yo live at No. 36, Main Road, Pyay. Write a letter to your cousin, Bo Bo, telling him about a terrifying dream you had last night and how you felt at that night. (10 marks)
You are Nyo Nyo. You live at No. 23, U Ba Gyan Street, Yangon. Write a letter to your friend, Kyi Phyu, telling her about the new supermarket which has been recently opened in your town and how grand it is.
You are Pa Pa and you live at No. 3, 37th street, Yangon. Write to your friend, Noe Noe, telling her about the recent trip you took with your friends and how much you enjoyed it.
In examining Section IX, Essay Writing, (See Example 9), the validity is especially high because this section measures the test-takers’ writing competence of the English language. As Heaton (1990, cited in Kuroki, 1996) notes, “[t]he only really satisfactory way to assess a student’s ability to write is by means of a composition test.”
Example 9: Example of Essay Writing Section of the Test
IX. Write an essay of three paragraphs on ONE of the following:(15 marks)
Unity is strength
A popular journal
The most boring season of the year
Compared with the writing tests of TOEFL (TWE) and IELTS (Writing section: Task 2), the differences are conspicuous. Consider the following sample essays:
Do you agree or disagree with the following statement:
People should sometimes do things that they do not enjoy doing.
Use specific reasons and details to support your answer.
(Good Luck TOEFL, 2012)
IELTS (Task 2)
In some countries children have very strict rules of behavior, in other countries they are allowed to do almost anything they want. To what extent should children have to follow rules?
Such sections test students’ writing ability and creativity since students synthesize meaningful concepts, reason critically, and rely less on formulaic responses and rote memorization. On the other hand, the type of essay writing of the English test of Myanmar university entrance exam merely assesses how test-takers can “package” their writing acumen within the topic as well as generate only grammatically correct sentences without demanding creativity. When considering washback, test-takers are likely to stick to translating from Myanmar to English instead of producing and performing in English. Undoubtedly, they would focus on producing only grammatically correct sentences.
Viewpoints and suggestions made by teachers and students
Having examined the writing section of the test, the following interview subjects discussed teaching and preparation for the Essay and Letter Writings Sections of the test. Many teachers admit that students have mediocre (if not poor) writing skills. While they are proficient at grammar, they are unable to apply the theory of writing to the actual practice of it. Consequently, they find that students are reluctant to do the writing tasks and are used to learning passages by heart and rewriting them from memory. Students often believe that learning composition wastes time since they do not need any creative writing for the exam. A teacher with over 36 English teaching experience here points out:
The students today don’t know how to construct a sentence easily and no questions are in the current test. For writing, students know and do only guided writing until 9th standard level, so it seems difficult for them to perform free writing at this level. (T5)
Most of the students learn the ready-made essays from the model essays book. A student, who is quite good at English, even mentioned that she has to learn the sample essays by heart. She also says that in writing essays, she uses her own ideas, and also some well-constructed sentences from the essays she memorized. Only two of the students interviewed practice essay writing by reading The New Light of Myanmar Journal (written in English) and one of them even studies Learner’s Choice Essay & Letter Practice Writing, GCE’O level, and IELTS books although they are aware that it is not for the test but rather to improve their writing skills. Teachers claim that only descriptive essays are tested at this high school levels although there are varieties of essay types such as narrative, argumentative, etc. T3 & T4 also reveal:
Here there’s a considerable point that due to this high-school level, only descriptive essays are assessed. It’d be better if different types of essays are given in the test which could measure student’s real writing ability. When the essay topics given in the test become repetitive, a student can get the same topic and write down his prepared essays. We also have difficulty in marking essays that the test-taker leant by heart rather than producing his own writing. (T3 & T4)
Moreover, the teachers interviewed disapprove of the testing requirements that state that essays are limited to three paragraphs with an unlimited number of words. A strict three-paragraph format stifles creative arguments, complex logic, and parallel reasoning – skills that differentiate truly exceptional writing from merely good. All the teachers interviewed prefer a word limit over a paragraph limit for this section.
At a minimum, because of differences and disparities among regions and multiple-version test creates doubt about the test’s ultimate reliability. A consistent, standardized test format works better than the current system. H. G. Widdowson and C. Criper (1975) note:
To know a language means to know something about how it fulfills communicative function … Until a learner knows how to use the resources of a grammar to send meaningful messages in real life situation, he can't be said to know the language (p. 129).
Needless to say, many valuable outcomes of language tests demand not only written but also oral tests that assess a student’s academic proficiency.
Hello , I am Gu Gu and I am from Myanmar country which is situated in Asia, near Thailand, China and India. Myanmar is one of the ASIAN countries. Now, I want to inform you all more about Myanmar as far as I can say.
I think most of the people in foreign countries don't know Myanmar. For me, I want to give messages about Myanmar and its culture. I am still a student and in Myanmar, we all speak in Myanmar, and so, I am not expert in English but I will try my best. Please correct me and reply to me how you think after you have read.
Our Myanmar country is also known as the land of pagodas. It is made up of seven states which are Kachin,Kayah,Kayin,Chin, Mon,Rakhine and Shan and seven divisions which are Mandalay,Yangon,Sagaing,Bago,Ayeyarwady,Tanintharyi and Magwe. There are many rivers, streams, beautiful beach shores and green woods. Myanmar is rich in national resources which are still unexplored.
Myanmar's population is estimated at over 47 million and it is made up of many different national races and religions. Most of the people are Buddhists and a few are other religions, Christian, Hindus,Isalam( I am sorry if spellings are wrong) and others. Most of the Myanmar people who are living in our country are honest, helpful, friendly, fun loving and peaceful. The ancient Myanmar people wear Longyi and Dhamai politely. But now, it does changes. The clothes and dresses are modernistic. I don't want to say about those clothes. The youths are influenced by the modernistic clothes. It is the current. I can understand that.
Myanmar has three seasons which are summer, Monsoon and winter (hot season, rainy season and cold season). It has temperate climate and it is also an agricultural country. Most of the people in Myanmar grow paddy and most important crops all the year.
And also, Myanmar people are fun-loving people. In Myanmar calendar, there are twelve months just like the Gregorian calendar which we are using today). Each month has its own festival and flower. The twelve months of Myanmar calendar are Tagu,Kason,Nayong,Waso,Wakaung,Tawthalin,Thadingyut ,Tansongmon,Nadaw,
Pyartho,Tabodwe,Tabaung. The most famous festivals are water festival in Tagu(April) which is also known as New Year festival( in Myanmar we call "Thingyan"), the water pouring festival at Banyan Tree in Kason ( May),the donating robe to the monks( the sons of Buddha) and the lightning festival in Thadingyut(October). These festivals are well known among some foreigners and the Myanmar people. The rest festivals are also well known too but only among the Myanmar people. I think each of this festival gives the Myanmar people pleasure, amusement, good-deed and unity. There are also many festivals which are still obscure and but celebrating in the rural areas. Myanmar has its own culture and tradition. Myanmar is also known as the golden land because of its glittering pagodas. However, I am very proud of my country although Myanmar is still a developing country. But I believe my country will develop one day.
Well, I think that's all about Myanmar but it is only a brief of Myanmar. Now, I think you all know Myanmar. (If you want know more about Myanmar, you can find in google or check this website http://www.myanmar.com . In conclusion, when I am writing this letter, the New Year festival (Thingy an) is still celebrating. I hope my letter will let you know more about Myanmar and I hope too. After you have read all, please give me some advice and commends about my country as well as about my English skills. To write about Myanmar, I have to read and find many facts to be exact. Thank you very much, everyone for your attention. I am Gu Gu from Myanmar. Be happy, everyone!