Unraveling Coreference Resolution in NLP!

Last Updated on February 25, 2021 by Editorial Team

Author(s): Ananya Banerjee

Natural Language Processing

Unraveling Coreference Resolution in NLP!

Coreference Resolution is one of the most essential Natural Language Processing (NLP) tasks. But, before we begin to understand Coreference Resolution, it is essential to understand the definition of discourse.

Discourse in the context of NLP refers to a sequence of sentences occurring one after the other. There will obviously be entities that are being talked about and possible references to those entities in the discourse. We use the word “mention” to refer to these references.

An example of a discourse:

Ana is a Graduate Student at UT Dallas. She loves working in Natural Language Processing at the institute. Her hobbies include blogging, dancing and singing.

Here, “Ana”, “Natural Language Processing” and “UT Dallas” are possible entities.

“She” and “Her” are references to the entity “Ana” and “the institute” is a reference to the entity “UT Dallas”.

Reference, in NLP, is a linguistic process where one word in a sentence or discourse may refer to another word or entity. The task of resolving such references is known as Reference Resolution. In the above example, “She” and “Her” referring to the entity “Ana” and “the institute” referring to the entity “UT Dallas” are two examples of Reference Resolution.

Let’s Summarize:

Discourse in the context of NLP refers to a sequence of sentences occurring one after the other

Reference is a linguistic process where one word in a sentence or discourse refers to another word or entity

The task of resolving such references is known as Reference Resolution.

Coreference Resolution in particular, is the process of resolving pronouns to identify which entities are they referring to. It is also a kind of Reference Resolution. The entities resolved may be a person, place, organization, or event.

Referent is the object that is being referred to. For example, “Ana” is the referent in the above example.

Referring expression are the mentions or linguistic expressions given in the discourse.

Two or more referring expressions that refer to the same discourse entity are said to corefer[1].

Now, let us look at another example to understand this better.

Example Discourse:

“Elon Musk was born on June 28, 1971. He is the founder, CEO , chief engineer and designer of SpaceX. The 49 year old is widely known as the mind behind Neuralink.”

Referring Expressions: Elon Musk, He, The 49 year old

Referent: Elon Musk

Corefering Expressions: {Elon Musk, He}, {Elon Musk, The 49 year old}

Now, that we understand the basics of coreference resolution, it is essential to understand what kind of references may exist in text. The knowledge about the kind of references helps us devise strategies to resolve them, if and when found.

References are usually of two kinds: Exaphor and Endophor. Endophor refers to an entity that appears in the discourse. While Exaphor refers to an entity that does not appear in the discourse.

Example of Endophor

Sentence: “Ana loves to read. She recently read a wonderful story.”

Here “She” refers to “Ana” which appears as a possible referrent that is mentioned explicitly in the discourse.

Example of Exaphor

Sentence: “Pick that up.”(pointing to an object)

Here “that” refers to a object which appears as a possible referrent for a object that it not mentioned explicitly in the discourse

There are primarily two kinds of Endophors: Anaphor and Cataphor. Anaphor refers to a situation wherein the referential entity or referent appears before its referencing pronoun in the discourse.

Example of Anaphor

Sentence: “Ana bought a dress. She loves it.”

Here “She” refers to “Ana” whose occurence precedes the occurence of its referencing pronoun “She” in the discourse.

While, Cataphor refers to a situation wherein the entity or referent occurs later than its referencing pronoun in the discourse.

Example of Cataphor

Sentence: “When she bought the dress, Ana didn’t know it was torn.”

Here “she” occurs before its referential entity or referent “Ana” in the discourse. Thus, this is an example of cataphor.

The set of corefering expressions is also called a coreference chain or a cluster. Now, that we understand what kinds of references are prevalent in literature, it is essential to grasp its linguistic properties as well.

Understanding these linguistic properties of the coreference relation helps us understand how to best perform coreference resolution and minimize the rate of error in the process[1]. The important thing to remember is that these properties may vary from one language to another depending upon its rules. So, please make sure that you are well versed in the grammatical rules of a language before attempting to perform coreference resolution. For the sake of lucidity in this article, we will consider English as our primary language.

Some of the linguistic properties which we will be talking about are:

Number and Gender Agreement

Recency

Grammatical Role

Verb Semantics

Selectional Restrictions

Repeated Mention

Parallelism

Now, let me explain these properties one by one. Number Agreement basically means that the referencing expressions should agree in number. While Gender Agreement implies that the referencing expressions agree in gender. Let’s look at an example of each.

Example of Gender and Number Agreement:

“Analisa works at Google. She loves her work.”

Here, “Analisa” and “She” agree in gender Female and they agree in number i.e., only one person “Analisa” works at Google and thus we use “She” to reference her rather than using other pronouns like “he”, “they”, etc. This is gender and number agreement.

Here, “horses” are plural. Hence, the referent used is “they” in order to refer to the entity “horses”.Thus, they agree on numbers. This is number agreement.

Another property to remember is the Grammatical role. This property takes advantage of the inherent grammatical nature of a sentence which gives more saliency value to subject entity as compared to an object entity. In other words, we assume that an entity which is a subject is usually more important than an object entity.

Example of Grammatical Role

“Ana works at an MNC with Tia. She usually works harder.”

In this sentence, we have “Ana” and “Tia” as candidate referents for the word “She”. Here “Ana” is the subject while “Tia” is the object. Thus, keeping in mind the saliency, we deem “Ana” to be coreferent to “She” rather than “Tia” .

The next factor to consider is Verb Semantics. Some verbs tend to lend more meaning to one of their arguments as compared to others while performing semantic analysis.

Example of Verb Semantics

“Ana helped Christa. She was the architect behind the project.”

“Ana condemned Christa. She was the architect behind the project.”

Here, in the first sentence, the usage of verb “helped” implies that the probability of Ana being the architect behind the project is higher than Christa. Thus, “She” refers to “Ana” in the first sentence.

However, in the second sentence, the usage of verb “condemned” implies that the probability of Christa being the architect behind the project is higher than Ana. Thus, “She” refers to “Christa” in the second sentence.

The next thing to understand is Selectional Restrictions. This utilizes semantic knowledge about a sentence to determine the referent’s preference.

Example of Selectional Restrictions

“I ate the roasted chicken in my pajamas after roasting it for three hours in the oven.”

Here, two possible referents for “it” are “pajamas” and “chicken”. The usage of the verb “eat” (“ate” is past tense for “eat”) implies that the referent entity must be edible, thereby choosing “chicken” as the referent for “it”.

Another essential feature to understand is Repeated Mention. This feature or property says that if an entity or a set of entities are referred to repeatedly in the discourse, then the probability of them being possible referents increases exponentially.

Example of Repeated Mention[2]

“John needed a car to get to his new job. He decided that he wanted something sporty. Bill went to the Acura dealership with him. He bought an Integra.”

Here, the repeated mention of “John” as compared to “Bill” as the focal point implies that “He” refers to “John” and not “Bill”.

Last but not the least, we talk about Parallelism. This property lends more importance to a referent if it can draw similar properties in terms of syntactic and semantic information from another sentence.

Example of Parallelism[2]

Mary went with Sue to the Acura dealership.

Sally went with her to the Mazda dealership.

Here, “her” refers to Sue as both the sentences imply similar synatactic and semantic structure and we can draw parallels between them.

Please do note that most of these factors are more valid while performing anaphoric reference resolution than other forms of reference resolution. In particular, the Exaphoric resolution of coreference is typically much harder to deal with and requires quite a different tactic.

Now, that you understand what is coreference resolution and how to resolve the references in case of ambiguity, let me also briefly mention two well-known libraries that can be used for coreference resolution. The first is StanfordCoreNLP and neuralcoref by Huggingface. You can use any of them to help ease your journey towards coreference resolution. You can also test these coreference systems online without any downloading necessary, here and here.

A few of the most well-known applications of coreference resolution can be found in Machine Translation, text, or natural language understanding tasks such as information extraction, question answering, summarization, etc.

I hope this article helped you better grasp the underlying concepts behind Coreference Resolution.

Thank you for reading!

References:

1. Speech and Language Processing, 3rd Edition by Dan Jurafsky and James H. Martin
2. Coreference Resolution and Discourse Coherence by Dr Mithun Balakrishnan

---

𝕌𝕟𝕣𝕒𝕧𝕖𝕝𝕝𝕚𝕟𝕘 ℂ𝕠𝕣𝕖𝕗𝕖𝕣𝕖𝕟𝕔𝕖 ℝ𝕖𝕤𝕠𝕝𝕦𝕥𝕚𝕠𝕟! was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Published via Towards AI