A standardized test is a test that is administered and scored in a consistent, or "standard", manner. Standardized tests are designed in such a way that the questions, conditions for administering, scoring procedures, and interpretations are consistent and are administered and scored in a predetermined, standard manner.
Any test in which the same test is given in the same manner to all test takers, and graded in the same manner for everyone, is a standardized test. Standardized tests do not need to be high-stakes tests, time-limited tests, or multiple-choice tests. The questions can be simple or complex. The subject matter among school-age students is frequently academic skills, but a standardized test can be given on nearly any topic, including driving tests, creativity, personality, professional ethics, or other attributes.
The opposite of standardized testing is non-standardized testing, in which either significantly different tests are given to different test takers, or the same test is assigned under significantly different conditions (e.g., one group is permitted far less time to complete the test than the next group) or evaluated differently (e.g., the same answer is counted right for one student, but wrong for another student).
Most everyday quizzes and tests taken by students typically meet the definition of a standardized test: everyone in the class takes the same test, at the same time, under the same circumstances, and all of the students are graded by their teacher in the same way. However, the term standardized test is most commonly used to refer to tests that are given to larger groups, such as a test taken by all adults who wish to acquire a license to have a particular kind of job, or by all students of a certain age.
Standardized tests are perceived as being fairer than non-standardized tests, because everyone gets the same test and the same grading system. This is fairer and more objective than a system in which some students get an easier test and others get a more difficult test. The consistency also permits more reliable comparison of outcomes across all test takers, because everyone is taking the same test. The prevalence of standardized testing in formal education has also been criticized for many reasons.
The definition of a standardized test has somewhat changed over time. In 1960, standardized tests were defined as those tests in which the conditions and content were equal for everyone taking the test, regardless of when, where, or by whom the test was given or graded. The purpose of this standardization is to make sure that the scores reliably indicate the abilities or skills being measured, and not other things, such as different instructions about what to do if the test taker does not know the answer to a question.
By the beginning of the 21st century, the focus shifted away from a strict sameness of conditions towards equal fairness of conditions. For example, a test taker with a broken wrist might write more slowly because of the injury, and it would be more fair, and produce a more reliable understanding of the test taker's actual knowledge, if that person were given a few more minutes to write down the answers to a most test. However, if the purpose of the test is to see how quickly the student could write, then this would become a modification of the content, and no longer a standardized test.
Main article: Imperial examination
The earliest evidence of standardized testing was in China, during the Han Dynasty, where the imperial examinations covered the Six Arts which included music, archery, horsemanship, arithmetic, writing, and knowledge of the rituals and ceremonies of both public and private parts. These exams were used to select employees for the state bureaucracy.
Later, sections on military strategies, civil law, revenue and taxation, agriculture and geography were added to the testing. In this form, the examinations were institutionalized for more than a millennium. Today, standardized testing remains widely used, most famously in the Gaokao system.
Standardized testing was introduced into Europe in the early 19th century, modeled on the Chinese mandarin examinations, through the advocacy of British colonial administrators, the most "persistent" of which was Britain's consul in Guangzhou, China, Thomas Taylor Meadows. Meadows warned of the collapse of the British Empire if standardized testing was not implemented throughout the empire immediately.
Prior to their adoption, standardized testing was not traditionally a part of Western pedagogy; based on the skeptical and open-ended tradition of debate inherited from Ancient Greece, Western academia favored non-standardized assessments using essays written by students. It is because of this, that the first European implementation of standardized testing did not occur in Europe proper, but in British India. Inspired by the Chinese use of standardized testing, in the early 19th century, British "company managers hired and promoted employees based on competitive examinations in order to prevent corruption and favoritism." This practice of standardized testing was later adopted in the late 19th century by the British mainland. The parliamentary debates that ensued made many references to the "Chinese mandarin system."
It was from Britain that standardized testing spread, not only throughout the British Commonwealth, but to Europe and then America. Its spread was fueled by the Industrial Revolution. The increase in number of school students during and after the Industrial Revolution, as a result of compulsory education laws, decreased the use of open-ended assessment, which was harder to mass-produce and assess objectively due to its intrinsically subjective nature. For instance, measurement error is easy to determine in standardized testing, whereas in open-ended assessment, graders have more individual discretion and therefore are more likely to produce unfair results through unconscious bias. When the score depends upon the graders' individual preferences, then the result an individual student receives depends upon who grades the test.
More recently, standardized testing has been shaped in part, by the ease and low cost of grading of multiple-choice tests by computer. Though the process is more difficult than grading multiple-choice tests electronically, essays can also be graded by computer. In other instances, essays and other open-ended responses are graded according to a pre-determined assessment rubric by trained graders. For example, at Pearson, all essay graders have four-year university degrees, and a majority are current or former classroom teachers.
Further information: List of standardized tests in the United States
Standardized testing has been a part of American education since the 1800s, but the widespread reliance on standardized is largely a 20th-century phenomenon. For instance the College Entrance Examination Board did not begin standardized testing in connection to higher education until 1900. This test was implemented with the idea of creating standardized admissions for the United States in northeastern elite universities. Originally, the test was also meant for top boarding school in order to standardize curriculum. With origins in World War I the Army Alpha and Beta tests developed by Robert Yerkes and colleagues. Before then, immigration in the mid-19th century contributed to the growth of standardized tests in the United States. Standardized tests were used in immigration when people first came over to test social roles and find social power and status.
Originally the standardized test was made of essays and was not intended for widespread testing. The College Board then designed the SAT(Scholar Aptitude Test) in 1926 for a broader IQ test. Notably, the Army IQ tests were what the first SAT test was based on in order to determine a student’s intelligence, problem solving skills, and critical thinking. In 1959, Everett Lindquist offered the ACT (American College Testing) for the first time. The ACT currently includes 4 main sections with multiple choice questions to test English, mathematics, reading, and science, plus an optional writing section.
Large population state testing began in the 1970s, and in the 1980s America began to assess nationally. In 2012, together 45 states is annual spending on assessments cost $27 per student and $669 million overall. However, once test involved administrative costs were included the cost per student increased to $1100. The need for the federal government to make meaningful comparisons across a highly de-centralized (locally controlled) public education system has also contributed to the debate about standardized testing, including the Elementary and Secondary Education Act of 1965 that required standardized testing in public schools. U.S. Public Law 107-110, known as the No Child Left Behind Act of 2001, further ties public school funding to standardized testing. The goal of No Child Left Behind was to improve the education system in the United States by holding school and teachers accountable and attempting to close the educational gap between minority and non-minority children in public schools. Students' results on standardized tests were used to allocate funds and other resources such as teachers and administrators to schools. This policy does not provide a federal standard for schools, but allows each state to set their own standards. The Every Student Succeeds Act replaced the NCLB. It was signed into law by President Obama on December 10, 2015. This act was created in order to revise the provisions of the NCLC in order to further allow student achievement and success.
Standardized testing is a very common way of determining a student's past academic achievement and future potential. However, high-stakes tests (whether standardized or non-standardized) can cause anxiety. When teachers or schools are rewarded for better performance on tests, then those rewards encourage teachers to "teach to the test" instead of providing a rich and broad curriculum. In 2007 a qualitative study done by Au Wayne demonstrated that standardized testing narrows the curriculum and encourages teacher-centered instruction. As a result, standardized testing has become controversial in the United States.
An additional factor to consider in regards to standardized testing in the United States education system, is the socio-economic background of the students being tested. Research has shown that children from low-income and poor families do not receive the same emphasis on education from their parents as those students from higher income families. According to the Nation Center for Children in Poverty, 41 percent of children under the age of 18 fall into the category of lower income. (Kobal, H. and Jiang, Y., 2018) This is a large percent of the student population who start behind the learning curve and require specialized attention to get to where they need to be in order to perform well on the standardized test.
The Australian National Assessment Program – Literacy and Numeracy (NAPLAN) standardized testing was commenced in 2008 by the Australian Curriculum, Assessment and Reporting Authority, an independent authority "responsible for the development of a national curriculum, a national assessment program and a national data collection and reporting program that supports 21st century learning for all Australian students".
The testing includes all students in Years 3, 5, 7 and 9 in Australian schools to be assessed using national tests. The subjects covered in these testings include Reading, Writing, Language Conventions (Spelling, Grammar and Punctuation) and Numeracy.
The program presents students level reports designed to enable parents to see their child's progress over the course of their schooling life, and help teachers to improve individual learning opportunities for their students. Students and school level data are also provided to the appropriate school system on the understanding that they can be used to target specific supports and resources to schools that need them most. Teachers and schools use this information, in conjunction with other information, to determine how well their students are performing and to identify any areas of need requiring assistance.
The concept of testing student achievement is not new, although the current Australian approach may be said to have its origins in current educational policy structures in both the USA and the UK. There are several key differences between the Australian NAPLAN and the UK and USA strategies. Schools that are found to be under-performing in the Australian context will be offered financial assistance under the current federal government policy.
Design and scoring
Standardized testing can be composed of multiple-choice questions, true-false questions, essay questions, authentic assessments, or nearly any other form of assessment. Multiple-choice and true-false items are often chosen because they can be given and scored inexpensively, quickly, and reliably through using special answer sheets that can be read by a computer or via computer-adaptive testing. Some standardized tests have short-answer or essay writing components that are assigned a score by independent evaluators who use rubrics (rules or guidelines) and benchmark papers (examples of papers for each possible score) to determine the grade to be given to a response. Not all standardized tests involve answering questions; an authentic assessment for athletic skills could take the form of running for a set amount of time or dribbling a ball for a certain distance.
Most national and international assessments, however, are not fully evaluated by people; people are used to score items that are not able to be scored easily by computer (such as essays). For example, the Graduate Record Exam is a computer-adaptive assessment that requires no scoring by people except for the writing portion.
The term "normative assessment" refers to the process of comparing one test-taker to his or her peers. A norm-referenced test (NRT) is a type of test, assessment, or evaluation which yields an estimate of the position of the tested individual in a predefined population. The estimate is derived from the analysis of test scores and other relevant data from a sample drawn from the population. This type of test identifies whether the test taker performed better or worse than other students taking this test. A criterion-referenced test (CRT) is a style of test which uses test scores to show whether or not test takers performed well on a given task, not how well they performed compared to other test takers. Most tests and quizzes that are written by school teachers can be considered criterion-referenced tests. In this case, the objective is simply to see whether the student has learned the material.
Human scoring is relatively expensive and often variable, which is why computer scoring is preferred when feasible. For example, some critics say that poorly paid employees will score tests badly. Agreement between scorers can vary between 60 and 85 percent, depending on the test and the scoring session. Sometimes states pay to have two or more scorers read each paper; if their scores do not agree, then the paper is passed to additional scorers.
Open-ended components of tests are often only a small proportion of the test. Most commonly, a major academic test includes both human-scored and computer-scored sections.
|Student answers||Standardized grading||Non-standardized grading|
|Grading rubric: Answers must be marked correct if they mention at least one of the following: Germany's invasion of Poland, Japan's invasion of China, or economic issues.||No grading standards. Each teacher grades however he or she wants to, considering whatever factors the teacher chooses, such as the answer, the amount of effort, the student's academic background, language ability, or attitude.|
WWII was caused by Hitler and Germany invading Poland.
WWII was caused by multiple factors, including the Great Depression and the general economic situation, the rise of national socialism, fascism, and imperialist expansionism, and unresolved resentments related to WWI. The war in Europe began with the German invasion of Poland.
WWII was caused by the assassination of Archduke Ferdinand.
There are two types of standardized test score interpretations: a norm-referenced score interpretation or a criterion-referenced score interpretation.
- Norm-referenced score interpretations compare test-takers to a sample of peers. The goal is to rank students as being better or worse than other students. Norm-referenced test score interpretations are associated with traditional education. Students who perform better than others pass the test, and students who perform worse than others fail the test.
- Criterion-referenced score interpretations compare test-takers to a criterion (a formal definition of content), regardless of the scores of other examinees. These may also be described as standards-based assessments, as they are aligned with the standards-based education reform movement. Criterion-referenced score interpretations are concerned solely with whether or not this particular student's answer is correct and complete. Under criterion-referenced systems, it is possible for all students to pass the test, or for all students to fail the test.
Either of these systems can be used in standardized testing. What is important to standardized testing is whether all students are asked equivalent questions, under equivalent circumstances, and graded equally. In a standardized test, if a given answer is correct for one student, it is correct for all students. Graders do not accept an answer as good enough for one student but reject the same answer as inadequate for another student.
The considerations of validity and reliability typically are viewed as essential elements for determining the quality of any standardized test. However, professional and practitioner associations frequently have placed these concerns within broader contexts when developing standards and making overall judgments about the quality of any standardized test as a whole within a given context.
In the field of evaluation, and in particular educational evaluation, the Joint Committee on Standards for Educational Evaluation has published three sets of standards for evaluations. The Personnel Evaluation Standards was published in 1988, The Program Evaluation Standards (2nd edition) was published in 1994, and The Student Evaluation Standards was published in 2003.
Each publication presents and elaborates a set of standards for use in a variety of educational settings. The standards provide guidelines for designing, implementing, assessing and improving the identified form of evaluation. Each of the standards has been placed in one of four fundamental categories to promote educational evaluations that are proper, useful, feasible, and accurate. In these sets of standards, validity and reliability considerations are covered under the accuracy topic. For example, the student accuracy standards help ensure that student evaluations will provide sound, accurate, and credible information about student learning and performance.
In the field of psychometrics, the Standards for Educational and Psychological Testing place standards about validity and reliability, along with errors of measurement and issues related to the accommodation of individuals with disabilities. The third and final major topic covers standards related to testing applications, credentialing, plus testing in program evaluation and public policy.
Importance of testing
Standardised testing is considered important and these tests do assess what is taught on the national level. They are used to measure objectives and how schools are meeting educational state standards.
There are three primary reasons for Standardized tests: Comparing among test takers, Improvement of ongoing instruction and learning, and Evaluation of instruction.
Considering the information presented above, students undergoing the testing have been told to not spend copious amounts of their own time to study and prepare for the tests, although students believe they need to do well to ensure they don't let down their school.
Standardized tests put large amounts of pressure on students. Some children who are considered at the top of their class choke when it comes to standardized tests such as the citywide.
Reflection of testing
Parents and community activates around the country explain that the education system are failing student. Standardized testing is included in efforts to improve the education system. Standardized testing gives a detailed account of how student improvement and teach effectiveness are evaluated, which can show how the school effectiveness sits on a national scale.
Standardized testing is used as a public policy strategy to establish stronger accountability measures for public education. While the National Assessment of Education Progress (NAEP) has served as an educational barometer for some thirty years by administering standardized tests on a regular basis to random schools throughout the United States, efforts over the last decade at the state and federal levels have mandated annual standardized test administration for all public schools across the country.
The idea behind the standardized testing policy movement is that testing is the first step to improving schools, teaching practice, and educational methods through data collection. Proponents argue that the data generated by the standardized tests act like a report card for the community, demonstrating how well local schools are performing. Critics of the movement, however, point to various discrepancies that result from current state standardized testing practices, including problems with test validity and reliability and false correlations (see Simpson's paradox).
Critics also charge that standardized tests encourage "teaching to the test" at the expense of creativity and in-depth coverage of subjects not on the test. Multiple choice tests are criticized for failing to assess skills such as writing. Furthermore, student's success is being tracked to a teacher's relative performance, making teacher advancement contingent upon a teacher's success with a student's academic performance. Ethical and economical questions arise for teachers when faced with clearly underperforming or underskilled students and a standardized test.
Critics also object to the type of material that is typically tested by schools. Although standardized tests for non-academic attributes such as the Torrance Tests of Creative Thinking exist, schools rarely give standardized tests to measure initiative, creativity, imagination, curiosity, good will, ethical reflection, or a host of other valuable dispositions and attributes. Instead, the tests given by schools tend to focus less on moral or character development, and more on individual identifiable academic skills.
- Offers Guidance to Teachers. Standardized tests will allow teachers to see how their students are performing compared to others in the country. This will help them revise their teaching methods if necessary to help their students meet the standards. 
- Allows Students to See Own Progress. Students will be given the opportunity to reflect on their scores and see where their strengths as well as weaknesses are. 
- Provide Parents Information about their Child. The scores can allow parents to get an idea about how their child is doing academically compared to everyone else of the same age in the nation. 
- Let's Government Know What Areas Need to be Improved. Tests that are taken by everyone can help the government determine where students are struggling the most. With this information, they can implement solutions to fix the issue, allowing students to learn and grow in an academic environment. 
One of the main advantages of standardized testing is that the results can be empirically documented; therefore, the test scores can be shown to have a relative degree of validity and reliability, as well as results which are generalizable and replicable. This is often contrasted with grades on a school transcript, which are assigned by individual teachers. It may be difficult to account for differences in educational culture across schools, difficulty of a given teacher's curriculum, differences in teaching style, and techniques and biases that affect grading. This makes standardized tests useful for admissions purposes in higher education, where a school is trying to compare students from across the nation or across the world. Examples of such international benchmark tests include the Trends in International Mathematics and Science Study (TIMMS) and the Progress in International Reading Literacy Study (PIRLS). Performance on these exams have been speculated to change based on the way standards like the Common Core State Standards (CCSS) line up with top countries across the world.
There are three metrics by which the best performing countries in the TIMMS (the "A+ countries") are measured: focus, coherence, and rigor. Focus is defined as the number of topics covered in each grade; the idea is that the fewer topics covered in each grade, the more focus can be given to each topic. The definition of coherence is adhering to a sequence of topics covered that follows the natural progression or logical structure of mathematics. The CCSSM was compared to both the current state standards and the A+ country standards. With the most number of topics covered on average, the current state standards had the lowest focus. The Common Core Standards aim to fix this discrepancy by helping educators focus on what students need to learn instead of becoming distracted by extraneous topics. They encourage educational materials to go from covering a vast array of topics in a shallow manner to a few topics in much more depth.
Standardized tests also remove teacher bias in assessment. Research shows that teachers create a kind of self-fulfilling prophecy in their assessment of students, granting those they anticipate will achieve with higher scores and giving those who they expect to fail lower grades.
Another advantage is aggregation. A well designed standardized test provides an assessment of an individual's mastery of a domain of knowledge or skill which at some level of aggregation will provide useful information. That is, while individual assessments may not be accurate enough for practical purposes, the mean scores of classes, schools, branches of a company, or other groups may well provide useful information because of the reduction of error accomplished by increasing the sample size.
Opponents claim that standardized tests are misused and uncritical judgments of intelligence and performance, but supporters argue that these aren't negatives of standardized tests, but criticisms of poorly designed testing regimes. They argue that testing should and does focus educational resources on the most important aspects of education — imparting a pre-defined set of knowledge and skills — and that other aspects are either less important, or should be added to the testing scheme.
Disadvantages and criticism
- Validity, efficacy, and predictive power. Many contend that overuse and misuse of these tests harms teaching and learning by narrowing the curriculum. According to the group FairTest, when standardized tests are the primary factor in accountability, schools use the tests to narrowly define curriculum and focus instruction. Accountability creates an immense pressure to perform and this can lead to the misuse and misinterpretation of standardized tests. FairTest says that negative consequences of test misuse include narrowing the curriculum, teaching to the test, pushing students out of school, driving teachers out of the profession, and undermining student engagement and school climate. Critics say that "teaching to the test" disfavors higher-order learning. While it is possible to use a standardized test without letting its contents determine curriculum and instruction, frequently, what is not tested is not taught, and how the subject is tested often becomes a model for how to teach the subject.
- Uncritical use of standardized test scores to evaluate teacher and school performance is inappropriate, because the students' scores are influenced by three things: what students learn in school, what students learn outside of school, and the students' innate intelligence. The school only has control over one of these three factors. Value-added modeling has been proposed to cope with this criticism by statistically controlling for innate ability and out-of-school contextual factors. In a value-added system of interpreting test scores, analysts estimate an expected score for each student, based on factors such as the student's own previous test scores, primary language, or socioeconomic status. The difference between the student's expected score and actual score is presumed to be due primarily to the teacher's efforts.
- Some teachers would argue that Standardized Test only measures a student’s current knowledge and it does not reflect the students progress from the beginning of the year. A result created by individuals that are not apart of the student's regular instruction, but by professionals that determine what students should know at different ages. In addition, teachers agree that the best test creator and facilitator are themselves. They argue that they are the most aware of students abilities, capacities, and necessities which would allow them to take a longer on subjects or proceed on with the regular curriculum.
- Notable Opponents. In her book, Now You See It, Cathy Davidson criticizes standardized tests. She describes our youth as "assembly line kids on an assembly line model," meaning the use of the standardized test as a part of a one-size-fits-all educational model. She also criticizes the narrowness of skills being tested and labeling children without these skills as failures or as students with disabilities. Widespread and organized cheating has been a growing culture in today's reformation of schools.
- Education theorist Bill Ayers has commented on the limitations of the standardized test, writing that "Standardized tests can't measure initiative, creativity, imagination, conceptual thinking, curiosity, effort, irony, judgment, commitment, nuance, good will, ethical reflection, or a host of other valuable dispositions and attributes. What they can measure and count are isolated skills, specific facts and function, content knowledge, the least interesting and least significant aspects of learning." In his book, The Shame of the Nation,Jonathan Kozol argues that students submitted to standardized testing are victims of "cognitive decapitation." Kozol comes to this realization after speaking to many children in inner city schools who have no spatial recollection of time, time periods, and historical events. This is especially the case in schools where due to shortages in funding and strict accountability policies, schools have done away with subjects like the arts, history and geography; in order to focus on the contest of the mandated tests.
- Testing Minorities. Monty Neill, the director of the National Center for Fair and Open Testing, claims that students who speak English as a second language, who have a disability, or who come from low-income families are disproportionately denied a diploma due to a test score, which is unfair and harmful. In the late 1970s when the graduation test began in the United States, for example, a lawsuit delayed that many Black students had not had a fair opportunity on the material they were tested on the graduation test because they had attended schools segregated by law. “The interaction of under-resourced schools and testing most powerfully hits students of color”, as Neill argues, “They are disproportionately denied diplomas or grade promotion, and the schools they attend are the ones most likely to fare poorly on the tests and face sanctions such as restructuring.” 
- In the journal The Progressive, Barbara Miner explicates the drawbacks of standardized testing by analyzing three different books. As the co-director of the Center for Education at Rice University and a professor of education, Linda M. McNeil in her book Contradictions of School Reform: Educational Costs of Standardized Testing writes “Educational standardization harms teaching and learning and, over the long term, restratifies education by race and class.” McNeil believes that test-based education reform places higher standards for students of color. According to Miner, McNeil “shows how test-based reform centralizes power in the hands of the corporate and political elite-- a particularly frightening development during this time of increasing corporate and conservative influence over education reform.” Such test-based reform has dumbed down learning, especially for students of color.
- On a student and educator level. There is criticism from students themselves that tests, while standardized, are unfair to the individual student. Some students are "bad test takers", meaning they get nervous and unfocused on tests. Therefore, while the test is standard and should provide fair results, the test takers are at a disadvantage, but have no way to prove their knowledge otherwise, as there is no other testing alternative that allows students to prove their knowledge and problem-solving skills.
- Some students suffer from test anxiety. Test anxiety applies to standardized tests as well, where students who may not have test anxiety regularly feel immense pressure to perform when the stakes are so high. High stakes standardized testing includes exams like the SAT, the PARCC, and the ACT, where doing well is required for grade passing or college admission.
- Standardized tests are a way to measure the education level of students and schools on a broad scale. From Kindergarten to 12th grade, students participate in required test taking. In that amount of time, the average student takes 112 standardized tests, which equates to about 10 tests per year. At this rate, the average amount of testing takes about 2.3% of total class time. Although standardized tests were designed to improve the education system, they are creating many negative effects on students and teachers.
Standardized tests have caused the quality and depth of the educational curriculum to diminish (Rooks, Noliwe, and Noliwe Rooks). Instead of teachers developing a curriculum that addresses the needs of the actual students in their classrooms, they end up using the required material which they did not take any part in creating. The required material often contains pacing guidelines which regulate when substance should be taught and scripted lessons which often limits the teacher’s abilities to make relevant decisions in a classroom. The tests have narrowed the curriculum to a lot of schools, usually squeezing out classes such as art and music simply because they are excluded in the tests, then they are wiped out of the curriculum. Teachers are then forced to teach subjects that only influence the literacy level and comprehension ability of a student and leave out the ones that often require talent or skill.
Standardized testing places a lot of stress and pressure on children and teachers. Teachers are put under a lot of stress because the better students do on the test the more federal funding that school and district will receive. This causes teachers to teach to the test rather than teach to the life skills children will use and need. In some cases, schools have shortened or removed recess so that more time can be spent preparing and practicing for the standardized tests. The pressure of this and the removal of a stress outlet, recess, means that children, along with teachers, are going to become depressed and sleep-deprived. Being depressed and sleep-deprived causes children to act out more than usual which places more stress on the teachers. Teachers do not get the results back until the end of the summer which means they will not be able to use those results to help those children because they will already be on to the next grade. Standardized tests place an unnecessary amount of stress on teachers and students without yielding any information in a timely manner.
- Standardized testing puts pressure not only on students, but on teachers as well. New Jersey Governor Chris Christie has proposed educational reform in New Jersey that pressures teachers not only to "teach to the test," but also have their students perform at the potential cost of their salary and job security. The reform calls for performance-based pay that depends on students' performances on standardized tests and their educational gains. However, students vary based on cognitive, developmental, and psychological abilities, so it is unfair to teachers with students with difficulties on the test.
In an April 1995 "meta-analysis" published in the Journal of Educational and Psychological Measurement, Todd Morrison and Melanie Morrison examined two dozen validity studies of the test required to get into just about any Masters or PhD program in America: the Graduate Record Examination (GRE). This study encompassed more than 5,000 test-takers over the past 30 years. The authors found that GRE scores accounted for just 6 percent of the variation in grades in graduate school. The GRE appears to be "virtually useless from a prediction standpoint," wrote the authors. Repeated studies of the Law School Admissions Test (LSAT) find the same. The SAT's maker, the Educational Testing Service (ETS), now claims the SAT is not an "aptitude" test but rather an assessment of "developed abilities."
Finally, standardized tests are not inexpensive. It has been reported that the United States spends about 1.7 billion dollars annually on these tests. In 2001, it was also reported that only three companies (Harcourt Educational Measurement, CTB McGraw-Hill and Riverside Publishing) design 96% of the tests taken at the state level.
Test scores are in some cases used as a sole, mandatory, or primary criterion for admissions or certification. For example, some U.S. states require high school graduation examinations. Adequate scores on these exit exams are required for high school graduation. The General Educational Development test is often used as an alternative to a high school diploma.
Other applications include tracking (deciding whether a student should be enrolled in the "fast" or "slow" version of a course) and awarding scholarships. In the United States, many colleges and universities automatically translate scores on Advanced Placement tests into college credit, satisfaction of graduation requirements, or placement in more advanced courses. Generalized tests such as the SAT or GRE are more often used as one measure among several, when making admissions decisions. Some public institutions have cutoff scores for the SAT, GPA, or class rank, for creating classes of applicants to automatically accept or reject.
Heavy reliance on standardized tests for decision-making is often controversial, for the reasons noted above. Critics often propose emphasizing cumulative or even non-numerical measures, such as classroom grades or brief individual assessments (written in prose) from teachers. Supporters argue that test scores provide a clear-cut, objective standard that minimizes the potential for political influence or favoritism.
The National Academy of Sciences recommends that major educational decisions not be based solely on a single test score. The use of minimum cut-scores for entrance or graduation does not imply a single standard, since test scores are nearly always combined with other minimal criteria such as number of credits, prerequisite courses, attendance, etc. Test scores are often perceived as the "sole criteria" simply because they are the most difficult, or the fulfillment of other criteria is automatically assumed. One exception to this rule is the GED, which has allowed many people to have their skills recognized even though they did not meet traditional criteria.
- ^Sylvan Learning glossary, retrieved online, source no longer available
- ^Popham, W.J. (1999). "Why standardized tests don't measure educational quality". Educational Leadership. 56 (6): 8–15.
- ^Phelps, Richard P. "Role & Importance of Testing". nonpartisaneducation.org. Retrieved 2016-05-17.
- ^ abcOlson, Amy M.; Sabers, Darrell (October 2008). "Standardized Tests". In Good, Thomas L. 21st Century Education: A Reference Handbook. SAGE Publications. pp. 423–430. doi:10.4135/9781412964012.n46. ISBN 9781452265995.
- ^"Chinese civil service". Encyclopædia Britannica. Retrieved 2 May 2015.
- ^ abcdeMark and Boyer (1996), 9–10.
- ^ abKazin, Edwards, and Rothman (2010), 142.
- ^Rich, Motoko (2015-06-22). "Grading the Common Core: No Teaching Experience Required". The New York Times. ISSN 0362-4331. Retrieved 2015-10-06.
- ^Darity Jr, William. "International Encyclopedia of the Social Sciences". Encyclopedia for Background Information. Gale Cengage Learning. Retrieved 25 January 2017.
- ^Gould, S. J., "A Nation of Morons", New Scientist (6 May 1982), 349–352.
- ^Johnson, Robert. "Standardized Tests." Encyclopedia of Educational Reform and Dissent. SAGE Publications, INC. 2010. 853–856.Web.
- ^Garrison, Mark J. A Measure of Failure: The Political Origins of Standardized Testing. Albany: State University of New York, 2009. Print.
- ^Darity Jr, William. "International Encyclopedia of the Social Sciences". Encyclopedias for Background Information. Gale Cengage Learning. Retrieved 25 January 2017.
- ^Fletcher, Dan. "Standardized Testing." Time. Time Inc., 11 Dec. 2009. Web. 09 Mar. 2014.
- ^"What's on the ACT." ACT Test Sections. N.p., n.d. Web. 05 May 2014
- ^Stiggins, Richard (2002). "Assessment Crisis: The Absence Of Assessment FOR Learning"(PDF). Phi Delta Kappan.
- ^Strauss, Valerie (March 11, 2015). "Five Reasons Standardized Testing Isn't Going to Let Up". The Washington Post. The Washington Post. Retrieved 26 January 2017.
- ^"History and Background of No Child Left Behind". Bright Hub Education9 June 2015. Web. 12 October 2015. http://www.brighthubeducation.com/student-assessment-tools/3140-history-of-the-no-child-left-behind-act/
- ^"No Child Left Behind." – Education Week Research Center. N.p., 19 Sept. 2011. Web. 06 July 2014. <http://www.edweek.org/ew/issues/no-child-left-behind/>. "Problems With Standardized Testing." Education.com. N.p., 3 November 2013. Web. 01 July 2014. <http://www.education.com/reference/article/Ref_Test_Problems_Seven/>.
- ^Au, Wayne (2007-06-01). "High-Stakes Testing and Curricular Control: A Qualitative Metasynthesis". Educational Researcher. 36 (5): 258–267. doi:10.3102/0013189X07306523. ISSN 0013-189X.
- ^Claiborn, Charles. "High Stakes Testing". Encyclopedia of Giftedness, Creativity, and Talent. SAGE Publications, 2009. 9 April 2014.
- ^Kobal, H. and Jiang, Y., (2018) Basic Facts about Low Income Children. Retrieved from http://www.nccp.org/publications/pub_1194.html
- ^"Home – The Australian Curriculum v8.1". www.australiancurriculum.edu.au. Retrieved 2016-05-17.
- ^ETS webage about scoring the GRE.
- ^ abHoutz, Jolayne (August 27, 2000) "Temps spend just minutes to score state test A WASL math problem may take 20 seconds; an essay, 21⁄2 minutes". Seattle Times "In a matter of minutes, a $10-an-hour temp assigns a score to your child's test"
- ^Where We Stand: Standards-Based Assessment and Accountability (American Federation of Teachers) Archived August 24, 2006, at the Wayback Machine.
- ^"Joint Committee on Standards for Educational Evaluation". Joint Committee on Standards for Educational Evaluation. Retrieved 2 May 2015.
- ^Joint Committee on Standards for Educational Evaluation. (1988). The Personnel Evaluation Standards: How to Assess Systems for Evaluating Educators. Newbury Park, CA: Sage Publications.
- ^Joint Committee on Standards for Educational Evaluation. (1994). The Program Evaluation Standards, 2nd Edition. Newbury Park, CA: Sage Publications.
- ^Committee on Standards for Educational Evaluation. (2003). The Student Evaluation Standards: How to Improve Evaluations of Students. Newbury Park, CA: Corwin Press.
- ^"The Standards for Educational and Psychological Testing". http://www.apa.org. Retrieved 2 May 2015.
- ^Popham, W. James (April 2016). "Standardized Tests Purpose is the Point". Educational Leadership. 73 (7): 47.
- ^S., Hamilton, Laura; M., Stecher, Brian. "Standardized Tests Can Be Smarter | RAND". www.rand.org. Retrieved 2016-05-17.
- ^"NAEP Nations Report Card - National Assessment of Educational Progress - NAEP". nces.ed.gov. Retrieved 2018-02-19.
- ^Kohn, Alfie (2000). The Case Against Standardized Testing: Rising the Scores, Ruining the Schools. 361 Hanover Street Portsmouth,NH 03801-3912: Heinemann. ISBN 0325003254.
- ^ abc"Pros & Cons of Standardized Tests | Oxford Learning". Oxford Learning. 2014-10-29. Retrieved 2018-02-19.
- ^"Pros and Cons of Standardized Testing"(PDF). Columbia University. Spring 2013. Retrieved February 19, 2018.
- ^Kuncel, N. R.; Hezlett, S. A. (2007). "ASSESSMENT: Standardized Tests Predict Graduate Students' Success". Science. 315 (5815): 1080–81. doi:10.1126/science.1136618. PMID 17322046.
by Dr. John Poulsen and Kurtis Hewson
Standardized testing in some circles is demonized as the vilest form of assessment. These individuals point to many problems with how these tests are created and administered, as well how the results are used. In other circles standardized testing represents true assessment whereby individual performances can be compared to other performances in a meaningful manner. That is, standardized testing is seen by some as a fair form of comparison; others do not. Knowing where standardized testing came from and what were the motivations for its growth, may help in understanding and perhaps in being able to use the results of standardized tests to improve teaching and learning. This article serves as an overview of the history and current realities of standardized testing.
Considering the role standardized testing has acquired in education systems internationally, one can safely assume that a vast majority of Canadians have experienced these tests as students. More and more students’ lives are becoming influenced by standardized testing, as a societal push for educational accountability has led to a dramatic increase in the use of these assessments across districts and nations (Guskey & Jung, 2013). Their value is much debated by educators, academics, and politicians, but what is clear is that their use seems to be increasing rather than decreasing. Experiencing standardized tests as students can provide a useful perspective, however, it is important that faculty and students have a general understanding of the history of standardized or high-stakes testing, as well as a basic overview of the how these assessments are built.
This article will explore the history of standardized testing, recent developments within standardized testing, creation of test questions, and applicability.
Stiggins (2008) states that
these once-a-year tests are not likely to be of much value to classroom teachers as you plan and carry out day-to-day instruction. They are assessments OF learning that are too infrequent, broad in focus, and slow in returning results to inform the ongoing array of daily decisions. But this does not mean that these tests are without purpose or value. They can communicate valuable information about students’ achievement status to other decision makers (pp. 347-348).
This relatively rational statement could be considered a definition of the battle lines that have been drawn up between those who are proponents of standardized tests and those against them.
The intent in standardized testing is to have large numbers of students write a single test, then to compare any single score against all others to see how an individual’s score compares to the large sample. The results are then posted on a bell curve that indicates where a score sits within descriptive statistical standards. Standardized tests are given to large groups numbering at least in the thousands, sometimes millions. In order to make the results as valid as possible, thus “standardizing” the administration of the assessment, the tests are:
- written at the same time and same day for all students,
- administered with consistent instructions,
- allowed the same amount of time for each student to write the test, and
- scored in the same manner.
Scantron is a common method of marking bubble sheets of multiple-choice style questions. Essays are marked by specialists who have been trained to mark in similar fashion.
Burke (1999) maintains that traditionally “standardized” meant that the test is standard or the same in three ways: (a) format/questions, (b) instructions, and (c) time allotment. Format/questions means that the test questions are the same for all students writing the exam. The information that the students are to show they know is asked of them in the same format that is usually multiple choice. Multiple choice is the format of choice because as Stiggins (2008) suggests, “It is relatively easy to develop, administer, and score in large numbers” (p. 354). Further, in order for the test to be fair in the sense of all students having the same chance to answer each question correctly, all questions must be the same.
The instructions are to be the same as well. These are to be delivered in the same way to all students so that no students are advantaged or disadvantaged. The last standardization is time allotment. All students are to be given the same amount of time to finish the exam.
However, the standardization of standardized exams is being eroded. Common changes to standardized testing allow certain students to have more than the allotted amount of time. Some students with certain learning needs are now allowed to have more time than other students to complete the exam. These students are then often allowed to write in different rooms as well.
The second requirement of standardized tests is also frequently adapted. Students with reading problems can get “readers” to read the questions. The rationale behind this is that the curriculum asks that students know certain information. Whether the students know this information is the purpose of the exam, not whether the students can read. These readers may adapt the standardized instructions that the students receive. Also, reading the questions to the students may give them an advantage or disadvantage other students do not have. Therefore, the second and third requirements of standardized testing are no longer strongly in effect.
There are other forms of standardized testing that are available other than multiple-choice questions, for example, essay writing. This form of testing currently has the disadvantage of needing markers to assess the essays. Essay markers must be trained to gain a sense of what the standards are. Then they must engage in the time-consuming activity of reading the essays. Even with the training assessors can give significantly different grades to an essay.
Proponents of standardized testing point to large-scale use of the tests that go beyond the individual student or even the school. Standardized testing allows comparison between provincial education systems or even national education systems. Advocates say that standardized tests are impartial and rational. They state that standardized tests are an inexpensive way to check that schools and teachers are accountable, that students and therefore the public are getting the education that public dollars are paying for. Standardized tests by this measure are intended to examine the whole education system and therefore individual scores may be not as significant.
“… the standardization of standardized exams is being eroded.”
The history of standardized testing is underpinned by noble sentiments. Testing can be found in all cultures. Evaluating the understanding of someone learning a new skill is common for all societies. Standardized testing as we know it today began in earnest in China as a form of aptitude testing, trying to ascertain who would be best at a particular job. Fletcher (2009) states that, “The earliest record of standardized testing comes from China, where hopefuls for government jobs had to fill out examinations testing their knowledge of Confucian philosophy and poetry.” These exams started in about 100 CE but were firmly established during the Sui Dynasty in 605 CE. They attempted to predict aptitude by discerning the best candidates for the Chinese civil service.
The most recent impetus to standardized testing was the Industrial Revolution and the movement to increased schooling where students were moved out of the work force and into schools. One of the easiest and arguably the cheapest way to test large numbers of those children was with a standardized exam.
Alfred Binet (1857-1911) and Theodore Simon (1872-1961) developed what is now commonly known as an IQ Test, beginning in the late 1800s and culminating with the Binet-Simon scale in 1905. These intelligence tests were created in response to the French government wanting to develop special education classes for students who were not benefiting from the newly instituted regular compulsory education program. The tests tried to identify students who needed focused education in order to maximize their education. These standardized tests were an attempt to streamline education so that society would gain maximum benefit from each citizen, a noble sentiment.
The test contained problems arranged in order of difficulty in a range of subjects but had as the basis items assessing comprehension, reasoning, and judgment (Reynolds, Livingston, & Willson, 2009). Louis Terman (1877-1956), who was teaching at the time at Stanford University, noted the success of these exams and their potential applicability in America. He spearheaded the creation of the Stanford-Binet Test which remains, in its fifth iteration, the most popular IQ testing vehicle in existence.
Fletcher (2009) suggests that “… by World War I, standardized testing was standard practice: aptitude quizzes called Army Mental Tests were conducted to assign U.S. servicemen jobs during the war effort.” Robert Yerkes was one of the academics assigned to test the servicemen and then suggest appropriate placement. This testing of servicemen helped build up a record of statistical evidence for IQ testing. Carl Brigham worked with Yerkes in the testing of servicemen. After the war he published a book, A Study of American Intelligence, based on the results in World War I. From this finding and analysis he created the Scholastic Aptitude Test (SAT) in 1926. Its intention was to screen college applicants to insure the worthy candidates were allowed admission. The test became immediately popular and by 1945 it became a standard method of college and university entrance, again a noble enterprise.
Everett Linquist invented the American College Test (ACT) in 1959 as a competitor to the SAT. In 2011, more than 3.3 million individuals wrote SAT and ACT exams. The ACT is considered more of a test of accumulated knowledge while the SAT is thought to test logic. Other important standardized exams are the Medical College Admission Test (MCAT) and the Graduate Management Admission Test (GMAT).
These standardized tests that attempt to predict success or aptitude seem to be successful. Reynolds, Livingston, and Willson (2009) state, “As a general rule, research has shown with considerable consistency that contemporary intelligence tests are good predictors of academic success” (p. 334). Fishman and Pasanella (1960) reviewed SAT predictive validity in the 1950s, finding that the median correlation between student first-year success and the SAT score was a significant 0.61. Recently Kobrin, Patterson, Shaw, Mattern, and Barbuti (2008) found a correlation of 0.29, a respectable correlation between SAT scores and First Year Grade Point Average (FYGPA).
In Alberta, standardized testing began in the 1960s. McEwen (1995) suggests that Alberta’s introduction of achievement testing for Grades 3, 6, and 9 was done in response to a worldwide wave of educational reform that wanted more accountability in education. At the Grade 12 level, diploma exams were reinstated in 1984 after being removed for a few years. McEwen clarifies the reason for the achievement tests:
Public education is funded by taxpayers who want and have a right to know if they are getting value for their investment. Such accountability requires public information. An indicator system is a tool to focus reform and to improve accountability by providing better information about the education system’s performance. The goals, or intended benefits, of implementing indicator systems are to assess the effectiveness and efficiency of the educational enterprise, to improve education, and to provide a mechanism for accountability (p. 28).
Pros and Cons of Standardized Testing
The primary conundrums in standardized testing of achievement lie in the validity and applicability of the test results. Validity relates to how accurately the test results actually reflect the students’ knowledge about the subject. Standardized tests use a minimum number of questions and getting even one or two wrong due to environmental reasons will affect the individual student’s results. The factors that affect a student getting a question right or wrong may be infinite and could be organized into (a) situational/environmental confounding factors, (b) personal/emotional factors, and (c) grade-spread requirement in standardized testing.
Even though standardized testing attempts to minimize confounding variables by requiring students to write in similar situations, it may be that some students are writing in situations that are significantly different from other students, for example, it might be too bright or too dark or even too cold or too hot. The testing conditions may cause students to perform poorly such as when students might miss questions not because they do not know the material but for something as simple as the testing centre had poor lighting that caused headaches in students, or because the testing room was too cold and did not allow certain students to focus.
Students who are poor test takers because of nerves associated with tests may not be able to show what they can accomplish in the high-stakes atmosphere of standardized testing. Their anxiety becomes the determining factor of how well they do the test, not whether they know the material. Even students who are normally good test takers can have a skewed result; for example, a student who had an emotional moment just before the test might not be able to focus and receives a result that is not reflective of his or her capabilities.
Perhaps the primary concern with achievement standardized testing is that testing should be based on curricular outcomes that are mandated by the provincial or state governing bodies. Standardized tests have to make a one-size-fits-all test that will not fit all because as Popham (1999) says, “… standardized achievement tests will invariably contain a number of items that are not aligned with what’s emphasized in a particular setting” (p. 331). A 1983 study of alignment between textbook content and the standardized test found that, “In no case was even 50 percent of a test’s content satisfactorily addressed in any textbook” (Popham, p. 331). That is, there was a poor correlation between what was in the test and in the textbooks that were a prime resource to prepare students for the test.
Test creators seek a score spread in their questions. They seek questions that are not answered correctly by too many students. Questions that are answered correctly by more than 60% of the students are usually removed from the test. Popham indicates this is a problem because “… items on which students perform well often cover the content that, because of its importance, teachers stress” (p. 332). So the important material that is required by the curriculum is often not tested.
How questions are determined to be most worthy for standardized testing is important. When deciding which questions to use, test creators, in essence, try to find questions that only the top 50% of the students will get right. These types of questions are popular in standardized testing because they support the common theory of testing whereby the highest achieving students answer the questions correctly. So, standardized tests can be self-affirming. Students who are in the top 50% of the class answered the questions correctly because they are in the top 50% of the class.
Further, if a concept is taught to all students in a class and all students answer the question correctly, that question will not be used in the future as it does not spread the students’ scores so that fine-grained norm-referenced numbers can be associated with each student. That is, if all students did well on the test then there would be no bell curve and the associate connection with where each student sits on the curve. Put more simply, there have to be questions that are only answered by about 50% of the students in order for comparisons to be made.
A student’s socio-economic status is highly correlated to standardized achievement test scores. This is probably due to the tests being skewed to reflect learning that children gain at home. Again there is a curriculum and testing mismatch. For example, if a question asks about a “field of work” such as law or medicine, students whose parents are in such professions may understand the concept from conversations at home. However, students whose parents work in the service industry or work at the local grocery store may not. Answering the question correctly may not be a function of what was learned at school but rather what has been learned out of school. Antagonists to standardized achievement testing suggest that it is not fair to check on student achievement that is not in the curriculum.
What instructors or textbooks focus on may not be reflected in the test. The requirement for a score spread in the exams means that questions that are answered by a majority of students will probably be removed because they do not discriminate enough.
The history of standardized testing suggests that the impetus for large-scale testing has been based on noble aspirations, primarily that of having the right person in the right place, whether that place is the correct job in the military or the correct form of education. Standardized testing has value in today’s society. Aptitude testing for admission into colleges and universities seems to be especially effective as quantitative research has established links between such testing and later success at post-secondary institutions.
Achievement testing has issues especially related to situational/environmental factors, personal/emotional factors, and grade-spread requirement that may make applicability difficult to ascertain. That is, standardized testing may be best at determining aptitude or future ability in an individual and also good at examining a school district’s efficaciousness. Standardized tests seem to be weaker at being able to correctly indicate how much a specific student has learned.
>> Back to Table of Contents
Alberta Assessment Consortium (2012). A new look at public assurance: Imagining the possibilities for Alberta students. Retrieved from http://www.aac.ab.ca/a-new-look-at-public-assurance-imagining-the-possibilities-for-alberta-students.html
Alberta Education (1997). Teaching Quality Standard applicable to the provision of basic education in Alberta. (Ministerial order #016/97). Retrieved from http://education.alberta.ca/media/6734948/teaching%20quality%20standard%20-%20english.pdf
Bew, Lord P. (2011). Independent review of key stage 2 testing, assessment and accountability, final report, as written for the Department of Education. Retrieved from https://www.education.gov.uk/publications/standard/publicationDetail/Page1/DFE-00068-2011
Boardman, A. G., & Woodruff, A. L. (2004). Teacher change and “high stakes” assessment: What happens to professional development. Teaching & Teacher Education, 20(6), 545-557.
Booi, L., & Couture, J. C. (2011). Testing, testing. What Alberta can learn from Finland about standardization and the role of the teacher. Alberta Views, 7, 28-32.
Brookhart, S. M. (2001). The “Standards” and classroom assessment research. Paper presented at the annual meeting of the American Association of Colleges for Teacher Education, Dallas, TX. (ERIC Document Reproduction Service No. ED451189).
Burke, K. (1999). The mindful school: How to assess authentic learning (3rd ed.). Arlington Heights, IL: Skylight Publishing.
Fishman, J. A., & Pasanella, A. K. (1960). College admission selection studies. Review of Educational Research, 30(4), 298-310.
Fletcher, D. (2009, December 11). Standardized testing. Time. Retrieved from http://www.time.com/time/nation/article/0,8599,1947019,00.html
Franklin, C. A., & Snow-Gerono, J. L. (2007). Perceptions of teaching in an environment of standardized testing: Voices from the field. The Researcher, 21(1), 2-21.
Gordon, S. P., & Reese, M. (1997). High-stakes testing: Worth the price? Journal of School Leadership, 7, 345-368.
Gronlund, N., & Waugh, C. (2009). Assessment of student achievement (9th ed.). Upper Saddle River, NJ: Pearson.
Guskey, T. R., & Jung, L. A. (2013). Answers to essential questions about standards, assessments, grading, & reporting. Thousand Oaks, CA: Corwin.
Kobrin, J., Patterson, B., Shaw, E., Mattern, K., & Barbuti, S. (2008). Validity of SAT for predicting first year college grade point average (Report No. 2008-5). New York, NY: College Board. Retrieved from http://professionals.collegeboard.com/profdownload/Validity_of_the_SAT_for_Predicting_First_Year_College_Grade_Point_Average.pdf
McEwen, N. (1995). Accountability in education in Canada. Canadian Journal of Education, 20, 1-17.
Pedulla, J. P. (2003). State-mandated testing – What do teachers think? Educational Leadership, 61(3), 42-46.
Popham, J. (2002). Classroom assessment: What teachers need to know (3rd ed.) Boston: Allyn and Bacon.
Popham, W. J. (1999). Why standardized tests don’t measure educational quality. Educational Leadership, 56(6), 8-15.
Reynolds, C., Livingston, R., & Willson, V. (2009). Measurement and assessment in education (2nd ed.). Upper Saddle River, NJ: Pearson.
Stiggins, R. J. (1999). Are you assessment literate? The High School Journal, 6(5), 20-23.
Stiggins, R. J. (2008). An introduction to student-involved assessment for learning (5th ed.). Columbus, OH: Pearson Merrill Prentice Hall.
>> Back to Table of Contents