Robust Gram Embeddings

Taygun Kekec; David Tax

Robust Gram Embeddings

Taygun Kekec, David Tax

Pattern Recognition and Bioinformatics

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

2 Citations (Scopus)

Abstract

Word embedding models learn vectorial word representations that can be used in a variety of NLP applications. When training data is scarce, these models risk losing their generalization abilities due to the complexity of the models and the overfitting to finite data. We propose a regularized embedding formulation,
called Robust Gram (RG), which penalizes overfitting by suppressing the disparity
between target and context embeddings. Our experimental analysis shows that the RG model trained on small datasets generalizes better compared to alternatives, is more robust to variations in the training set, and correlates
well to human similarities in a set of word similarity tasks.

Original language	English
Title of host publication	Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing
Publisher	Association for Computational Linguistics
Pages	1060-1065
Number of pages	6
Publication status	Published - 2016
Event	EMNLP 2016: Conference on Empirical Methods in Natural Language Processing - Austin, TX, United States Duration: 1 Nov 2016 → 5 Nov 2016

Conference

Conference	EMNLP 2016
Country/Territory	United States
City	Austin, TX
Period	1/11/16 → 5/11/16

Cite this

@inproceedings{43edf6c16ca6465a8f88330ef618f1ae,

title = "Robust Gram Embeddings",

abstract = "Word embedding models learn vectorial word representations that can be used in a variety of NLP applications. When training data is scarce, these models risk losing their generalization abilities due to the complexity of the models and the overfitting to finite data. We propose a regularized embedding formulation,called Robust Gram (RG), which penalizes overfitting by suppressing the disparitybetween target and context embeddings. Our experimental analysis shows that the RG model trained on small datasets generalizes better compared to alternatives, is more robust to variations in the training set, and correlateswell to human similarities in a set of word similarity tasks.",

author = "Taygun Kekec and David Tax",

year = "2016",

language = "English",

pages = "1060--1065",

booktitle = "Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing",

publisher = "Association for Computational Linguistics",

note = "EMNLP 2016 : Conference on Empirical Methods in Natural Language Processing ; Conference date: 01-11-2016 Through 05-11-2016",

}

TY - GEN

T1 - Robust Gram Embeddings

AU - Kekec, Taygun

AU - Tax, David

PY - 2016

Y1 - 2016

N2 - Word embedding models learn vectorial word representations that can be used in a variety of NLP applications. When training data is scarce, these models risk losing their generalization abilities due to the complexity of the models and the overfitting to finite data. We propose a regularized embedding formulation,called Robust Gram (RG), which penalizes overfitting by suppressing the disparitybetween target and context embeddings. Our experimental analysis shows that the RG model trained on small datasets generalizes better compared to alternatives, is more robust to variations in the training set, and correlateswell to human similarities in a set of word similarity tasks.

AB - Word embedding models learn vectorial word representations that can be used in a variety of NLP applications. When training data is scarce, these models risk losing their generalization abilities due to the complexity of the models and the overfitting to finite data. We propose a regularized embedding formulation,called Robust Gram (RG), which penalizes overfitting by suppressing the disparitybetween target and context embeddings. Our experimental analysis shows that the RG model trained on small datasets generalizes better compared to alternatives, is more robust to variations in the training set, and correlateswell to human similarities in a set of word similarity tasks.

M3 - Conference contribution

SP - 1060

EP - 1065

BT - Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing

PB - Association for Computational Linguistics

T2 - EMNLP 2016

Y2 - 1 November 2016 through 5 November 2016

ER -

Robust Gram Embeddings

Abstract

Conference

Fingerprint

Cite this