Design and Implementation of Translation Testing Procedures

By Alaina Brantner, MATI Member

 

Regardless of their size, for translation firms providing services in multiple languages, the likelihood that none of the professionals in the organization or department read the language of many of the target deliverables is great. Firms therefore rely on translators from around the world—most of whom their vendor and project managers will never meet in person—for a translation product in a language that no one within their organization can read. When the product is in a language that no one can read, developing well-designed processes to mitigate against the risks associated with working with unqualified providers is critical. These risks include (but are not limited to):

Ultimately, these risks bear on the reputations of firms (not to mention the health and safety of end users) and their ability to protect their relationships with all their clients. A deliberate translator onboarding process is designed to prevent these kinds of outcomes, with layers of preventions in place to ensure the quality of the product for a firm and its clients. A well-developed translator testing procedure is an essential component of that overall preventative process.

Translator Testing – The Cost Not to Test

Collectively, any system for translator onboarding consists of three main stages: initial screening, testing, and a probationary period. (For more on the initial screening stage that takes place prior to testing, see my article, “Key Components of Successful Translator Recruitment” in The ATA Chronicle.) Firms invest a significant amount of time and money at the initial screen stage to pass qualified candidates on to testing, and testing represents the next level of investment made by firms in the onboarding process. Not only an additional layer of protection safeguarding the integrity of the product, testing also helps firms ensure that future investments of time and resources are allocated toward developing mutually beneficial relationships with the best talent available. Any candidate’s movement through the overall onboarding process correlates to increasing investments of time and resources that a firm makes in onboarding and training that candidate. The farther candidates have progressed, the greater the losses for firms when they are determined to not meet organizational requirements. Any candidates who do not meet these requirements during live jobs represent the greatest potential losses to firms, not only in the investments made to date in that candidate, but in terms of rework on live jobs and the resulting risks to client relationships.

The testing phase does not eliminate the risks associated with selling a product that has been contracted for in an online environment in a language the professionals in an organization do not read. Testing does, however, help prevent firms from making great investments of time and resources in candidates that threaten the well being of firms, through lack of subject-matter knowledge, or neglect of best practices. Testing increases the likelihood that the finite resource investments of a firm are being allocated to building relationships and training the candidates that best meet an organization’s needs.

Strategies for Translation Test Design

The primary focus of any translation test is the verification of a translator’s language and subject-matter capabilities. Content for translation testing is developed based on the domain in which a firm provides translation services. For example, within the technical fields, tests might include a variety of specialized terminology, while organizations that focus on marketing might utilize texts with idioms and other turns of phrase for a sample of translators’ creative problem-solving strategies. Beyond language and subject-matter-related verifications, the most successful tests will draw upon the experiences of the various stakeholders of the translation and production processes of a firm. For example, based on feedback from reviewers and desktop publishers on each of their respective stages, testing can be designed to account for recurring issues—such as the inconsistent translation of references to headers in the body of a manual, or the placement of formatting tags. This gives vendor managers the opportunity to gauge candidates’ abilities in a range of areas and to better learn if a candidate is a good fit for their organization.

While translation testing affords firms a valuable opportunity to check translators in a variety of areas, when establishing the grading structure for testing, language fluency and subject matter expertise are ultimately the most critical skills to test. As recommended in the ASTM's “Standard Guide for Quality Assurance in Translation,” organizations will ideally work with a trusted language lead when carrying out the native language editing of any translation samples or tests.[i] This editor checks the translation first and foremost for accuracy and completeness, and during this first pass review, any of certain categories of objective errors, such as mistranslations, automatically disqualify a candidate.

Once the translation has been determined to be accurate and complete, the editor and/or a non-native proofreader can proceed to check the target content for other stylistic issues. Depending on the field in which an organization provides translation services, consistency will be a major factor. The ability to follow instructions is also key, so reviewers need to check that candidates have adhered to guidelines for the treatment of proper nouns, abbreviations, and conversions. Language-specific conventions can also be verified, to ensure that non-breaking spaces have been used before colons in Canadian French, for example, or that titles are capitalized to meet target language norms.

Beyond style checks, translation testing is also a firm’s first opportunity to verify candidates’ technological capabilities, including work with more complex file formats and within the computer-assisted translation (CAT) environment. This technology component should not be overlooked. Often, to win work, less qualified providers make big claims about use of and comfort with CAT technology. By including something as simple as a CAT-compatible glossary of a few key terms, vendor managers can quickly determine if a candidate’s capabilities meet their technological claims.

Finally, budgets are an important factor to consider in the design of translation tests, and firms use a variety of strategies to manage the costs associated with testing. I personally argue against relying on free tests to make the crucial determinations on candidates’ capabilities outlined in this article. Firms that test potential translators by having them translate live jobs for free are advised to carefully consider the repercussions of this strategy. The adage “you get what you pay for” speaks volumes about the results one can reasonably expect from this sort of an approach. To adjust for the skewed results of this strategy, firms are either failing more candidates or lowering the quality expectations of their testing system. Either adjustment is detrimental to a firm’s well-being. The former fails to capitalize on a firm’s investments at the initial screening stage, which is especially costly assuming that for every translator passed on to linguistic testing, 12.5 must be contacted, as was true of my 2016 recruitment efforts.[ii] The latter introduces vulnerability into a firm’s onboarding processes, because unqualified candidates are given greater chances of passing. Overall, the expectation for free services is also an ineffective means through which to positively embark on the mutually-beneficial relationships with the highly-qualified professionals that help to safeguard the integrity of a firm’s product and reputation.

Translator Testing – A Controlled Environment

Deliberate testing is developed based on an understanding of a wide variety of factors, including industry standards, the circumstances under which translation is carried out, and the objectives that are met through strong testing procedures. Fundamental to the design of testing is an awareness that firms rely on recommendations made by outside providers for languages that often none of the professionals in their organization read. This awareness must be balanced with realistic expectations for the outcome of the overall onboarding process. No set of actions can account for every eventuality. Still, based on experience and knowledge, procedures can be established and updated to set firms up for the greatest chance of success in their onboarding efforts, and ultimately for the translation services they are able to provide as a result. Carefully designed translation testing is a key component of the overall preventative onboarding process, in which issues with great potential costs for live jobs are identified and overcome in a low-cost, controlled environment, before they can affect a firm’s reputation and its relationships with its end clients.

-----

Alaina Brantner is a translation consultant specializing in project, vendor, and translation memory management. A Spanish> English translator, she has an MA in language, literature and translation from the University of Wisconsin, Milwaukee. She served as a director of the Midwest Association of Translators and Interpreters (an ATA chapter) from 2012 to 2016. Contact: alaina.brantner@gmail.com.

[i] Brantner, Alaina. “Key Components of Successful Translator Recruitment.” The ATA Chronicle (American Translators Association), May/June 2017, 22.

[ii] ASTM International, F 2575-06, “Standard Guide for Quality Assurance in Translation” (June 2006), 6-7.