Au Pair USA by InterExchange
English Language Skills Observations 2021 - 2023
A Case Study Using Three Years Of English Proficiency Testing Of Au Pairs
Academic Language Solutions had the privilege of testing the English language skills of au pair candidates on the Au Pair USA J1 exchange visitor program (aupairusa.org). Prospective au pairs were given the iTEP Au Pair test before their acceptance into the Au Pair USA program. The iTEP Au Pair Test is designed to measure the speaking and listening skills of test-takers in a familiar atmosphere, without the pressures imposed by a formal test center. The exams can be taken online, on-demand, in the comfort of a test-taker’s home or school environment.
Au pair candidates must be prepared to communicate well with conversational skills from Day One of their experience in the USA. Thus, their speaking and listening skills must be carefully assessed in an objective and professional manner before they can be accepted as an au pair. The Department of State, which oversees this exchange program, sets minimal standards for screening candidates, allowing program sponsors to interview candidates and having administrative staff with no professional English screening credentials determine their individual qualifications for acceptance into the program. Au Pair USA by InterExchange decided to have candidates professionally and objectively screened for the program and chose iTEP, the International Test of English Proficiency, to assess the English language skills of their au pair candidates.
The iTEP Au Pair test is not meant to qualify au pairs for their functional responsibilities. The test objectively evaluates English proficiency to keep children safe and ticks the “speaks English well” box on the au pair’s application. It is not meant to replace the required interview by the sponsor’s staff, which focuses on intent and au pair qualifications. It is intended to relieve the interviewer from assessing English skills, a task they are likely not fully qualified for unless they have experience as an English as a Second Language instructor.
Assessing English Proficiency vs. Assessing Program Intent
Au Pairs are recruited by international exchange organizations in various countries worldwide. These organizations are responsible for pre-screening the candidates and preliminarily assessing their language skills and intent for participation in the program. In this case, The program sponsor, Au Pair USA by InterExchange, is responsible for accepting qualified young people into the program.
Academic Language Solutions, directed by individuals with experience sponsoring State Department-designated exchange programs, has selected the iTEP exam to best assess the skills required to successfully perform functions and adapt to an exchange visitor program.
Academic Language Solutions has found that organizations that employ standardized tests to screen participants’ English proficiency will, in the long run, recruit increasingly better-qualified applicants for their programs. This may be because overseas agents who recruit for these exchange programs recruit for multiple sponsors. The candidates who are most prepared and confident to take standardized tests like the iTEP Au Pair Test are likely their best candidates for the program. If this is the case, Academic Language Solutions believes that candidate strength will improve yearly, with the best candidates being assigned to the sponsor with the most stringent requirements.
What has the experience been like with the candidates tested for the Au Pair USA program?
Did the scores of applicants in the au pair program improve year over year? iTEP overall scores measure aggregate key English language skill sets, including listening and speaking skills, for the au pair program. We looked at scores from over 800 au pair exams taken by Au Pair USA candidates from 2021 to 2023.
While it can’t be proven that overall scores improved due to a regimen of standardized tests, it can’t be denied that the scores did, for some reason, improve. This is not likely because the English skills of all speakers have improved. On the contrary, research has shown that overall, the English skills of international students have lessened.
The EF English Proficiency Index (EPI) is the world’s largest ranking of countries and regions by English language skills. Recent testing of 2.2 million young people tracked their proficiency in English skills and compared results yearly. They concluded that young people’s English language skills were, in fact, decreasing across the globe. The largest rate of decrease occurred among young (18-20) women, a demographic that is close to the demographic of au pair applicants on cultural exchange programs.
In our study of au pair candidates in the Au Pair USA research, 96% of the applicants were female in a similar age range as referred to in the EF research. With global scores going down, why did scores among the Au Pair USA applicants go up? Could it be that agents selected better applicants to be presented to the sponsor or that applicants prepared better for their onboarding process, knowing they needed to succeed on a stringent standardized English test?
CEFR is the world's most widely accepted indicator of language skills today. CEFR (Common European Framework of Reference) identifies levels of language skills ranging from beginner (A1) to mastery (C2). We assigned a numeric value to the CEFR levels attained by Au Pair USA candidates over the three-year period. We saw growth in this indicator as well.
iTEP tests can track not just overall performance but achievements in sub-skills as well. For example, with au pairs, listening is an essential skill. Au Pairs must be able to understand instructions given by host parents. The safety of children is at stake. It matters if a child is given a teaspoon versus a tablespoon of medicine. It matters if a child is given one versus two pills. And it matters if a child is tucked into bed at 7 pm versus 9 pm.
Did Au Pair USA by InterExchange get au pairs who were increasingly skilled in listening?
Listening skills improved significantly between Year One and Year Two, but still slightly overall between Year Two and Year Three. We would expect skills to level off in subsequent years but still stay elevated as standardized tests were required and expected by candidates and agents.
While listening skills are paramount, the ability to communicate verbally is equally important. This is essential for au-pair-to-child communications and au-pair-to-parent communications. A vital element of the exchange program is the cluster concept, whereby au pairs engage in community activities with other au pairs. The language of this interaction is English, so speaking skills are also important.
Did Au Pair USA by InterExchange recruit better speakers year after year?
While listening skills and overall skills in English improved steadily, it is clear that speaking skills improved as well. This can only contribute to better overall performance of au pairs in the home and social environments.
Summary
Au Pair USA by InterExchange (aupairusa.org), one of the original au pair sponsors in the cultural exchange field, worked with Academic Language Solutions to test candidates utilizing the iTEP (International Test of English Proficiency) exam to assess listening and speaking skills. Academic Language Solutions conducted a research project to track the year-on-year change in pertinent skills of candidates applying to participate in the program. The theory being tested was that having a stringent testing regimen would attract candidates who were ultimately better suited to function as au pairs with families looking for a unique blend of cultural exchange and childcare skills.
Academic Language Solutions tested more than 800 candidates over a three-year period, tracing their listening and speaking skills. It was determined that the applicant pool improved in both listening and speaking skills each year.
Before introducing the use of the standardized test, the sponsor had used a candidate interview process to determine applicants' English language skills. Interviewers were generally not trained as English as a Second Language (ESL) professionals who would be comfortable assessing English language skills. While the interview process is essential in determining the intent and capabilities of the applicants, we believe that interviewers were distracted from this important task by being asked to measure language skills, a task for which they were neither trained nor prepared.
We believe it is essential for applicants in this childcare program to be ready to “hit the ground running” when they are caring for young children in the United States. This means being as capable as possible with their language skills, listening to and understanding instructions, and, in turn, communicating effectively in the spoken language.
We believe that having a stringent screening process with a standardized and proven English language test is one of the best ways to reach this goal. Our research with the program applicants proves that this process assures program sponsors that they have diligently screened candidates. We further believe that sponsors who use stringent standardized language testing can attest that they have made the best effort to provide host families with the most competent cultural exchange participants.
We are concerned that if this screening were to be paused, it would cause language proficiency to return to the pre-testing level of skills, which is clearly below the standards that have been achieved, as shown in the research project results. Standardized English language testing has shown a definite improvement in the English language skills of au pair candidates. Whether it is a causative improvement has yet to be determined, but it is an undeniable improvement.
Submitted by Randy LeGrant and Kevin Morgan
Academic Language Solutions
Related websites:
https://academiclanguagesolutions.com
February 8, 2024
Download This Case Study Today.
We only need your contact details to keep track of the downloads.