The 4 key criteria in assessing translation quality – and the questions to ask

translation quality assessment featured imageMost people don’t know how to go about assessing translation quality.

This article solves that – with a simple but remarkably effective 3-step process you can use to successfully evaluate the quality of almost any translation.

To get there, we first dissect what translation quality really means and identify the specific areas any translation must get right to be high quality.

This gives us 4 simple but essential questions which we wrap into our proven 3-step translation quality review process.

We think all clients and reviewers should know about this.

Contents
What is a high-quality translation?
– Our definition
– The 4 parameters/criteria to translation quality
Criterion 1: The role of meaning in assessing translation quality
– Equivalence of meaning – what is it?
– How different is different? – it’s not all black and white
– Our solution for the grey areas
– Three key tips for better translation quality assessment outcomes
– Translation Quality Assessment Question 1
Criterion 2: Evaluating a translation’s quality of expression
– Natural language
– Ambiguity
– Register/style
– Translation Quality Assessment Question 2
Criterion 3. Mistakes
– Translation Quality Assessment Question 3
Criterion 4. Consistency
– Translation Quality Assessment Question 4
Our 3-step translation quality assessment method
Step 1. Review for accuracy of meaning
Step 2: Review of wording
Step 3: Technical/formal check
Summary pdf (free download)

Video, slide show or article?
Choose your preferred version.

Video:

Slide-show:

Or continue with the article.


Defining translation quality – what is a high-quality translation?

Here’s our definition:

A high-quality translation is one that captures the meaning of the original text and is appropriately worded, free from error and internally consistent

Now this definition won’t work in every case or for every use or purpose clients may have for a translation. It’s not designed to – no definition ever will.

Rather, it’s a definition designed to be functional – to work in practice. It’s a framework reviewers can use to do what they’re asked to do every day – assess the quality of someone’s translation.

It will work for most business translation needs.

It’s a definition that requires some subjective judgement. Whether meaning has been fully captured and what constitutes appropriate wording won’t always be clear-cut, as we’ll see. There’s no way around that – any assessment of quality will be subjective to some degree.

And people will have different opinions. That’s something we need to accept – it’s simply the reality of dealing with a concept like “quality”.

 

Our definition identifies the 4 parameters that determine translation quality:

The 4 parameters to translation quality

– Meaning
– Wording / expression
– Errors
– Consistency

 

And the standards needed in each for a translation to be high-quality:

  • Equivalence of meaning
  • Appropriate wording / expression
  • No mistakes
  • Fully consistent

In summary:

Let’s flesh out what these mean in practice.

Criterion 1. The role of meaning in assessing translation quality

Meaning is a notoriously slippery concept.

As soon as we ask what something really means, things can get murky and unexpectedly complex. Academics have spent decades trying to nail it all down.

And yet, we all know what things mean, don’t we? We couldn’t communicate or understand each other if we didn’t.

It’s this practical, intuitive understanding of meaning the following discussion calls on. At this functional level it works, and works well.

 

Equivalence of meaning – what is it?

A high-quality translation will express all the meaning contained in the source text. And only that meaning.

In other words, it won’t omit, alter or add any meaning. This is what we typically mean when we say a translation is accurate.

Simple right? Mostly yes.

Firstly, it’s generally easy to identify when some meaning has been missed in a translation (omissions). And so too if it adds something that’s not in the original (additions).

That just leaves altered meaning – when the translation says something different to the original. This is often referred to as a mistranslation or translation mistake.

Most of these will also be obvious and easy to identify. So linguists will generally agree when meanings match or don’t match.

But not always.

 

How different is different? – it’s not all black and white

There are grey areas, and what one reviewer might see as a difference, another may not.

Take these two sentences:

Place the device on the skin and check the readout on the screen

Attach the device to the skin and check the readout on the screen

Someone focusing on the precise or literal meaning of the verbs “place” and “attach” might identify a difference in meaning.

But someone else might conclude the sentence is all about the device working and giving a readout, so this distinction isn’t important.

Anyone involved in evaluating translation quality will have many cases like this. Times when they need to decide if something is sufficiently different to no longer reflect the meaning of the original.

There’ll be times where they’re in two minds.

One approach to resolving such uncertainties is to focus on the key, underlying or essential meaning or message. Rather than the superficial or literal. So you’d ask “Does the translation capture the essential meaning/message of the original?”

Another approach is to assess how relevant or significant a potential difference in meaning is – is it a difference that matters?

Both approaches accept that some non-equivalence of meaning is OK, providing it’s not significant.

 

But is this OK?

Some people aren’t comfortable with this. They feel a translation should carry over all meaning, both essential and superficial, wherever possible.

And they don’t see it as the translator’s (or reviewer’s) role to decide what matters and doesn’t matter, or what’s important and what isn’t, in the original text.

But this can create a problem.

If we insist on all nuances of meaning being precisely and directly conveyed, we’re forcing the translator into a pretty literal translation.

And literal translations frequently don’t read well. They tend to use vocabulary and wording that isn’t natural in that language. So they can seem clunky and a bit awkward – which as we’ll see below won’t meet our definition of a high-quality translation.

So, there’s a dilemma:
If we want every nuance of meaning to be conveyed we’ll tick the “Equivalence of Meaning” box but may well fail at the “Quality of Expression” hurdle.

So what to do?

Our solution for the grey areas

This is how we see it:

1. Languages handle things differently, express ideas differently, view the world differently. Just like the cultures they reflect. As a result, meanings and vocabulary don’t always correspond precisely across languages.

2. A translation needs to respect the (target) language’s way of doing and saying things if it’s to be well-worded and read naturally.

3. This need for naturalness of wording means subtle variations in meaning will occur occasionally.

4. Some leeway is therefore needed when comparing shades and nuances of meaning. Expecting 100% exact (literal) equivalence isn’t always realistic and may well compromise quality of expression.

 

People won’t always agree

Judging translation accuracy and meaning equivalence is inevitably subjective. It’s often a value judgement.

So people will sometimes see things differently. They might disagree on both the extent of any differences and their importance.

A reviewer might identify a difference in meaning that the translator doesn’t see. This is best seen as different viewpoints or opinions, rather than black and white, right and wrong.

They’ll likely both have sound reasons for seeing things as they do.

 

Putting all this into perspective

This may be sounding rather complicated and ill-defined, but here’s the good news:

  • mostly it’ll be clear cut where a translation has captured the meaning of the original text and where it differs;
  • any differences of opinion on this should be few and far between – just the odd phrase or clause here and there, and
  • these will typically involve more subtle shades of meaning rather than the essential or main message of that section.

 

Three key tips for better translation quality assessment outcomes

1. Use experienced personnel for both your translation and any review process.
An experienced translator should automatically capture all meaning. And an experienced reviewer should make sound judgements on any grey areas and be less likely to make or request changes that others may not see as necessary.

2. Avoid getting embroiled in discussions over grey area issues.
These seldom reach a definitive conclusion – they’re a matter of opinion after all. And they can take up a lot of time – for you, the translator and reviewer.

3. Be crystal clear on who has final say on wording.
If the reviewer, we suggest they simply make any changes they think necessary. If the translator, the reviewer should flag any queries for the translator to consider.

 

This gives us the first of our four questions:

Translation Quality Assessment Question #1:
Does the translation have the same meaning as the original?


Criterion 2. Evaluating a translation’s quality of expression

We’ve said a high-quality translation will be “appropriately worded”. So what is “appropriate”?

It involves three aspects – naturalness, ambiguity and register/style.

a. Natural language

A high-quality translation will use natural sounding language, flow nicely and be easy to understand. There won’t be any awkward parts, and it generally won’t seem like a translation.

We call this good wording, high quality of expression, or fluency.

Only native speakers can really assess quality of expression. It requires their native speaker “feel”.

Generally native speakers will agree on what is and isn’t good wording.

But again, not always. Phrasing that is well worded to one linguist, may seem a little clunky to another.

This shouldn’t be a surprise. As we go through life we all develop connotations and emotional attachments around certain words and expressions – both positive and negative.

This leads to preferred ways of saying things, and other words and phrasing we like less. Because everyone’s life experiences are different, so too are our preferences.

In practice this means one linguist may occasionally consider some phrasing unnatural that another thinks is perfectly fine.

Again it’s not a matter of black and white, right and wrong, but rather reflections of individual wording preferences. Occasional differences in viewpoint on wording are natural and to be expected.

This is something that translation editors and reviewers need to bear in mind.

Do they change something in the translation because they prefer different wording, or do they accept the translator’s wording?

The problem with making those sorts of stylistic changes is that other people may not see them as improvements! Generally we recommend allowing the translator’s style and vocab preferences. In other words, only changing something if there is a strong reason for doing so.

 

b. Ambiguity

A sentence is ambiguous if it can be interpreted in different ways. An oft-quoted example is:

“I saw a man on a hill with a telescope”.

There are six reasonable interpretations here. They depend on who is on the hill (the writer or the man) and whether the telescope is being used by either or is installed on the hill.

Another example is President Bush’s response when asked about finding weapons of mass destruction in Iraq:

“But for those who say we haven’t found the banned manufacturing devices or banned weapons, they’re wrong, we found them”.

What does “them” refer to – the devices, the weapons or both?

Some potential interpretations will be unrealistic in context – like “saw a man” above meaning cutting a man with a saw.

So how does this relate to translations?

Generally, we don’t want a translation to be ambiguous and so open to misinterpretation. Especially not where the original wasn’t ambiguous and had a single clear meaning.

However, there are exceptions. Sometimes an ambiguity will be intentional, in which case the translation needs to reflect that. Perhaps as in President Bush’s response?

And in certain genres, writers may encourage reader interpretation – in poetry for example.

But for most business texts requiring translation, clients want clarity of meaning. Translations should reflect that and not introduce ambiguities.

c. Register and style

Register refers to the degree of formality or informality of vocabulary and phrasing.

For example, “ascertain” and “terminate” are more formal words than “find out” and “end”. And contractions like “isn’t” and “doesn’t” are more casual and informal than “is not” and “does not”.

Related is tone – the feelings, emotions or attitude conveyed by the words used.

Mostly we’ll want a translation to have similar tone and formality/informality to the original.

Style encompasses word choice and sentence and paragraph structure. For example, an ornate style might have long complex sentences, a sparse style much shorter sentences and be more direct and succinct.

It’s tempting to say a translation should have the same style as the original. But this may not always be the case. Languages and cultures sometimes use quite different styles for the same situation. Examples are in marketing texts and when dealing with sensitive subjects.

Better is to say the translation should use an appropriate style for the intended purpose or function of the text.

Here is our second question covering these three aspects:

Translation Quality Assessment Question #2:
Does the translation read like it should?


Criterion 3. Mistakes – a quality translation simply won’t have any

This is the easy one.

We’re not talking about translation mistakes (differences of meaning) here – these come under our first criterion above.

Rather we’re referring to the technical stuff – typos, punctuation, spelling, grammar errors. Obviously, a high-quality translation can’t have any of these.

There are accepted rules for these, so this is an objective rather than subjective criterion. Which means we should all agree on what’s a mistake and what isn’t.

And they’re easy to identify. They’ll stand out like a sore thumb to experienced reviewers – even when they’re not looking for them.
Plus, the spell checkers and other tools translators use will identify most (but not all).

Our third question is a simple one:

Question #3:
Are there any grammatical or spelling mistakes?


Criterion 4. Consistency

Our definition says a high-quality translation will be internally consistent.

Important here is consistency of vocabulary. Except in rare cases, we’d expect terms that are repeated in the original to be translated in the same way throughout the translation. Not doing this can confuse readers.

Related, we’d want proper names, dates and numbers handled consistently.

We’d also expect consistency of layout and formatting. This means fonts, fonts sizes, indents, spacing, bullets, numbering, capitalisation, etc.

Thirdly, we’d expect consistency of style and register.

But what if there are different styles within a document – shouldn’t the translation reflect that?

It depends.

A document might openly involve different writers using differing styles – a compilation of different chapters or sections penned by different people, for example. The translation would need to reflect the style of each and maintain consistency within that section or chapter.

Documents also often quote other more formal texts – for example legal clauses or formal reports. Here the translation should reflect the style of the quoted text.

But sometimes a document for translation will include parts copied from other documents with different styles. Resulting in a combination of more formal and less formal styles.

Here we say it’s best for the translation to adopt the document’s dominant style throughout, so everything is consistent.

Why?

Because it’ll be better received by the intended reader. Put another way, we don’t want a mishmash of styles in our translation detracting from the reader experience. So it’s in clients’ best interests.

It’s the same rationale for why we ensure consistency of formatting and layout even if the original text doesn’t. The more professional the translation, the better it’ll serve its intended purpose.

This gives us our final question:

Translation Quality Assessment Question #4:
Has the translation handled everything consistently?


Putting it all into practice: our simple but highly effective translation quality assessment method

We’ve identified the 4 questions we need to ask to judge a translation’s quality:

The 4 translation quality assessment questions

  • Does it capture the meaning of the original?
  • Does it read like it should?
  • Are there any grammatical or spelling mistakes?
  • Has it handled everything consistently?

 

But how do we do go about answering these?

Well, it’s too demanding to keep all 4 of these questions in mind and check for them at the same time. So we need to break the task up into more manageable chunks.

The ideal process involves three steps:

Step 1. Review for accuracy of meaning

The focus here is solely on meaning – with no other distractions.

We check that all meaning in the source text has been correctly carried over into the translation, with nothing extra added and nothing omitted.

This means systematically comparing the translation and original, phrase-by-phrase or sentence-by-sentence.

It’s more effective when done from hard copies. Things are much more easily missed when it’s done on screen.

We recommend placing the two texts side-by-side, covered over, and uncovering matching sections only as they’re being worked on. This keeps your focus solely on the text you’re comparing and minimises potential distractions.

It’s essential to work with short chunks of text – no more than 7 to 10 words at a time. Anything longer makes it hard to retain all the information in short-term memory. Meaning it’s much more likely you’ll miss a difference in meaning.

Each chunk of text should ideally be a discrete and complete unit of meaning. That’s because complete ideas are easier to retain in memory than incomplete ones. In practice this means we often work at phrase rather than sentence level.

For example, it will be more effective to break this earlier sentence into two chunks:

Each chunk of text should ideally be  /  a discrete and complete unit of meaning.

This will be easier than working with the whole sentence, where for example you might easily not notice if the concepts of “ideally” or “discrete” were missing in the translation.

And you wouldn’t want to divide the sentence up like this:

Each chunk of text should ideally be a discrete  /  and complete unit of meaning.

That’s because the first part is not a complete idea, so is difficult for our minds to handle and retain in memory.

Reviewing for accuracy of meaning is demanding work!

It requires intense concentration and is mentally taxing.

That’s why this step has a singular focus on meaning – it’s much more effective in picking up any inaccuracy than if you’re trying to concentrate on multiple things at once.

 

The additional benefit of focusing solely on meaning

Because of the intense focus at phrase and word level, reviewers tend to automatically notice any typos, grammar mistakes and inconsistency in vocabulary in this step.

This step answers question 1 – does the translation have the same meaning as the original?
And partially answers question 3 – are there any grammatical or spelling mistakes?
And partially answers question 4 – has everything been handled consistently?

 

Step 2: Review of wording

This step assesses the translation’s quality of expression and consistency of wording and style.

You need to be a native speaker of the language to judge quality of expression.

We’re now putting ourselves in the readers’ shoes to see how they’ll respond.

The reviewer reads through the translation looking out for anything that jars, seems unnatural or is hard to follow.

Anything that makes a reviewer pause should be revisited.

Practiced reviewers will also automatically pick up any inconsistency of vocab and style during this read through. With experience they develop a sort of sixth sense, an inbuilt radar, for anything that isn’t consistent.

Less experienced reviewers may not have this and need an additional pass through the translation – focusing particularly on how key vocabulary has been handled.

This step may also need to be split into two phases when dealing with more complex or lengthy texts.

This step answers question 2 – does the translation read like it should?
And partially answers question 4 – has everything been handled consistently?

 

Step 3: Technical/formal check

In this step the reviewer runs an eye over formatting and layout to ensure they are appropriate and consistent throughout.

They’ll also ensure there are no spelling or grammar errors, generally using the spell check function available in most programs.

They may also check proper name spellings, dates and numbers – a common source of errors.

This step answers question 3 – are there any grammatical or spelling mistakes?
And partially answers question 4 – has everything been handled consistently?

 


Conclusion

This method is a remarkably effective way of assessing translation quality across most business needs.

It works so well because it:

  • covers all the necessary bases
  • breaks the task down into manageable steps

The key is that each step looks at something different – so doesn’t overload the brain by trying to check too many things at once. That simply doesn’t work and is the perfect recipe for missing things.

It’s something of a tragedy that sound translation review methodologies aren’t widely known or used.

They’re seldom taught, either in translation degree courses or in the workplace.

We think all reviewers should follow this process, and recommend clients ensure any reviewers they engage adopt it.

The following summary pdf can be downloaded to help keep your review process on track.

Like your own summary of this article?

Let us quote for your next translation project.

We know our stuff and will deliver you high-quality work!

Simply:
– use our translation quote form, or
e-mail us the details
and we’ll be right back to you.

Other articles that may be of interest

The ultimate list of 51 different types of translation – and what they all mean


Realistic translation times – knowing how long a high-quality translation will take


The two essential steps for a successful translation review