Asset Details
Do you wish to reserve the book?
The Geometry of Distributed Representations for Better Alignment, Attenuated Bias, and Improved Interpretability
Hey, we have placed the reservation for you!
By the way, why not check out events that you can attend while you pick your title.
Oops! Something went wrong.
Looks like we were not able to place the reservation. Kindly try again later.
Are you sure you want to remove the book from the shelf?
The Geometry of Distributed Representations for Better Alignment, Attenuated Bias, and Improved Interpretability
Do you wish to request the book?
The Geometry of Distributed Representations for Better Alignment, Attenuated Bias, and Improved Interpretability
Please be aware that the book you have requested cannot be checked out. If you would like to checkout this book, you can reserve another copy
We have requested the book for you!
Your request is successful and it will be processed during the Library working hours. Please check the status of your request in My Requests.
Oops! Something went wrong.
Looks like we were not able to place your request. Kindly try again later.
Dissertation
The Geometry of Distributed Representations for Better Alignment, Attenuated Bias, and Improved Interpretability
2020
Overview
High-dimensional representations for words, text, images, knowledge graphs and other structured data are commonly used in different paradigms of machine learning and data mining. These representations have different degrees of interpretability, with efficient distributed representations coming at the cost of the loss of feature to dimension mapping. This implies that there is obfuscation in the way concepts are captured in these embedding spaces. Its effects are seen in many representations and tasks, one particularly problematic one being in language representations where the societal biases, learned from underlying data, are captured and occluded in unknown dimensions and subspaces. As a result, invalid associations (such as different races and their association with a polar notion of good versus bad) are made and propagated by the representations, leading to unfair outcomes in different tasks where they are used. This dissertation addresses some of these problems pertaining to the transparency and interpretability of such representations. A primary focus is the detection, quantification, and mitigation of socially biased associations in language representation. First, using the underlying geometrical structure of the high-dimensional space occupied by the representations, we build an understanding of the relative orientation between point sets. Next, we capture and isolate different feature subspaces or concept subspaces (such as gender or occupations in language representations) within the embedding space. Following this, we develop methods to decouple specific subspaces so as to better prepare the representations for specific downstream tasks. We also build a comprehensive range of probes to understand and highlight the different implicit associations learnt by the representations from underlying data, as an effort to distinguish between meaningful and invalid associations learnt and amplified by the embeddings. This is especially impactful as these representations drive much of the tasks and real world applications based on Natural Language Processing. Finally, we extend and apply these methods to different distributed representations and demonstrate the applications of increased interpretability.
