Test quality and usefulness โดย อังชรินทร์ ทองปาน

Classroom Achievement Test for Business English Students

I  Design statement

1.     Purpose of test

The purpose of the test is to inferences about reading and writing abilities which will be used as a part of evaluating the skill of reading comprehension and writing in business English at the end of the course. The content of achievement test generally based on the course syllabus. The test results are to be used to decide whether students do or do not exempt from study in English business I course and allowed to entered Business II course.

2.     Description of  TUL domains and task types

The test taker’s sole task is to read various kinds of business context and to write essay. The location is Business classroom.  Physical conditions are quiet and well lit. Materials and equipment are various kinds of authentic business reading text e.g. application letter, hotel reservation and service, company, etc and then writing prompt.

3.     Description of  characteristics of test takers

Personal of characteristics: The Test takers are male and female, aged 20 up, with second year university in English business students.

Topical Knowledge of test takers:  test takers are familiar with topic of business reading and basic reading and writing skills.

General level and profile of language ability:  intermediate to advance

4.     Define of constructs

The construct to be measured divided in to two skills as follow:

Part I Reading sub skills

1.      Predicting key points of information

      2.  Detailed reading : finding example

          3. Text analysis: way of giving example

          4. Scanning for specific items, information

          5. Inferring unstated information

          6. Skimming for gist, to locate key words

          7. Scanning to evaluate information, specific information

8. Identify a sequence of texts

          9. Inferring unstated information

          10. Texts analysis : way of expressing logical consequences

          11. Detailed reading : looking for logical conclusion

          12. Texts analysis : functions of main tenses; compound adjectives, non - defining

          relative clauses with : which and who, making contrasts

          13. Inferring unstated information, new stories from headlines, instead

          14. Identifying the writer’s attitude, unstated information

15. Transferring information from a text to a diagram

16. Retain chunks of language of different lengths in short-term memory

17. Recognize a core of words, and interpret word order patterns and their significance.

Part II   Writing essay

Organization; introductory paragraph, topic approach, topic sentence and supporting details.

Support ; support your point of view.

Clarify; your point of view clear

Usage and mechanics; complete sentences, spelling, capital letter, punctuation mark

5.     Plan for evaluating quality of usefulness

Reliability

It is the consistency of scores on each measurement or stability of the measurement results whether the result of the measurement points or of them. When measured results are able to highly believe assurance that is warranted, checking the results in many times, the consistent with the results is of the measurement itself. For my test, part I (reading) to collect evidence to for evaluating reliability, several examiners will give the same test to a group of test takers and calculate the consistency of scores across the different test of administration. Part II writing essay, several raters to score the same test and calculate the consistency of scores across the different rater.

Validity

Accuracy of the test is consistent with what you want to measure which is considered the most important feature of a test. In this test, I try to make a test to have validity especially the construct validity because I make the test from reading comprehension sub skills and writing skills

         Impact

The purpose from the test is reflect to the teacher, students and University. This means that the purpose of the test should be decided whether students do or do not meet minimum standard (60%) to be exempt Business English I course.

Practicality

It is good to separate the question and answer sheet it is easy to take the test. It is also about the necessary equipment will be available in each center and the there will be high degree for standardization in the test practicality. In my test,I have the test practicality because I separated the question and answer sheet and I typed the test-paper.

       Authentically

The purpose of the test is suitable for the real life of the takers. Therefore, the takers are able to take this examination in their daily life. It is quite important for test usefulness because the tester can do the best if the test linked to their experience and background. In my test, I choose the texts based on needs analysis about reading topic interested in the tester and the resources from the target language.

II Operationalization

Course description and amplified Objective

Course description

พัฒนาภาษาอังกฤษที่ใช้ในการสื่อสารทางธุรกิจ โดยเน้นฝึกทักษะการอ่าน และการ

เขียนจดหมายธุรกิจในรูปแบบต่างๆ เช่น จดหมายเชิญ จดหมายแสดงความยินดี จดหมายขอบคุณ จดหมายแสดงความเสียใจ จดหมาย สอบถาม ตอบสอบถามจดหมายสั่งซื้อ ตอบสั่งซื้อ การเขียนบันทึกข้อความ การประกาศและและโฆษณา

Principles of reading and writing business correspondence, letters of invitations,

letters of congratulations, letters of appreciations, letters of condolences, inquiries and requests, replies to inquiries and requests, orders and acknowledgement of orders, memorandums, notices and advertisements

Amplified Objective

Reading amplified objectives (Part A)

The learners have to read various kinds of business texts and to be able to identify main idea, details and sequence, transfer information from text, infer unstated information, noun referent and guess the vocabulary from context clue.

Writing amplified objectives (Part B)

To be able to identify the topic sentence and supporting details in the paragraph organization. The learners have to consider what the topic sentence and supporting details the write these sentence into paragraph. The topic sentence is the first sentence in paragraph form. These sentences are about “businessman”

Test of specification

Objective

Mode

Text type

Level of knowledge

Test techniques

Number of items

Time  (mins)

Weight percent

Scoring

M

Cp

T

Ct

E

Reading(Par I)

Word meaning 1. identify the meaning of the word in context.

R

Business Articles/ Business Document

/

/

/

M/C

5

9

13

0/1

Main idea 2. identify the main idea. 3.identify the topic sentence.

R R

Business Articles/Business Document

/

/

/ /

/

M/C M/C

3 5

5 9

8 13

0/1 0/1

Details 4.identify specific details.

R

Business articles

/

/

M/C

7

12

18

0/1

Reading objective

Mode

Text type

Level of knowledge

Test techniques

Number of items

Time  (mins)

Weight percent

Scoring

M

Cp

T

Ct

E

Reference 6. identify the pronoun . 7. identify the reference word in context. Inference 7. draw conclusion. 8. interpret the meaning 9  fact and opinion

R R

Business Articles/ Business Document Business Articles/ Business Document

/ /

/ / / /

/ / / /

/ /

M/C M/C M/C M/C M/C

5 4 6 4 1

9 7 10 7 3

13 10 15 10 5

0/1 0/1 0/1 0/1 0/1

Writing (Part B)

- to be able to identify the topic sentence and supporting details in the paragraph organization. - to write paragraph.

W

what is the good businessman should be”

/

/

-Write the paragraph

1 item (20 p)

20

20

Writing scale

Total

4

7

7

2

1

40

60

100

Answer key part I

NO.	A	B	C	D	NO.	A	B	C	D	NO.	A	B	C	D	NO.	A	B	C	D
1.					11.					21.					31.
2.					12.					22.					32.
3.					13.					23.					33.
4.					14.					24.					34.
5.					15.					25.					35.
6.					16.					26.					36.
7.					17.					27.					37.
8.					18.					28.					38.
9.					19.					29.					39.
10.					20.					30.					40.

Scoring criteria for writing (Part B)

Name	Communication (5)	Language (5)	Organization (5)	Appropriateness (5)	Total (20)
1. ..................... 2..................... 3..................... 4..................... 5..................... 6..................... 7..................... 8..................... 9..................... 10.....................

Scoring Rubric for writing (Part B)                                

16-20    (Excellent)   - very good paragraph, good for language organization and  appropriateness

11-15    (Good)  -  can understand, some few mistake of language organization and appropriateness

6-10     (Fair) )  -  can understand, but many problem with language organization and  appropriateness

 0-5       (Improve) )  -  do not understand, should improve of language organization and appropriateness

III Administration, try out and use

Administration

A preparing the testing environment will be prepared by the business teacher staff at the English business department, arrange the meeting with the test administrators at the testing centre. Make sure they understand them and answer questions they may have.

Communicating the instructors by have the test administer read the instruction aloud while the test takers look at them in their test booklet. For this step, make sure that the testing center will keep the proctor on hand and answer the questions or testing procedure.

Try out

             For this step, the test was to administered for 30 students and then correct the data  by considering  the item facility between  .20 – 80  and item discrimination .20 up.  Choose the items selected to find out the reliability whole test by using formula KR 20 = .80

Use

Business English teacher sent test to academic affairs and meeting to score test