Self-supervised Learning for Graph-to-Text Generation from Biomedical Graphs

Primary supervisor

Ehsan Shareghi

Knowledge graphs (KGs) play an important role in Natural Language Processing (NLP) and store information in a structured and machine-accessible way. They are used in different domains and fields. Generating texts from KGs is an important NLP task which transforms graph into natural language. For example, given a subgraph from a KG, we aim to get a corresponding description. Texts are easier to understand for human than graph-structured data. The project will involve leveraging cutting edge technologies such as Transformers [1,2] and developing self-supervised learning mechanisms [3] to facilitate learning from the unlabelled data. The work will be submitted as a publication to top-tier NLP venues, in late 2023.

Student cohort

Single Semester
Double Semester

URLs/references

[1] BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model

[2] SciFive: a text-to-text transformer model for biomedical literature

[3] Self-supervised Graph Masking Pre-training for Graph-to-Text Generation

Required knowledge

  • Proficiency in Python is required
  • Working knowledge of NLP is required
  • Familiarity with the gensim and pytorch libraries is desired
  • Familiarity with Text-based Transformer models and HuggingFace is desired
  • Very good verbal and written communication skill is required