keyword
biomedical NLP
Biomedical natural language processing, commonly abbreviated as biomedical NLP or BioNLP, is a specialized subfield of artificial intelligence and computational linguistics focused on developing computational methods to understand, analyze, extract, and generate text within the healthcare and life sciences domains. Unlike general language processing, it is tailored to handle the distinct linguistic complexities of medical and scientific communication, such as specialized terminology, ambiguous abbreviations, and intricate biological relationships. By utilizing domain-specific datasets and adapted language models, biomedical NLP powers critical applications across scientific literature and clinical records, including named entity recognition, relation extraction between entities like drugs and diseases, information retrieval, medical question answering, and text generation to support clinical decision-making and scientific research.
2 items
BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers
Ran Xu, Wenqi Shi, Yue Yu, Yuchen Zhuang, Yanqiao Zhu, May Dongmei Wang, Joyce C. Ho, Chao Zhang, Carl Yang
Why you should read this
Presents BMRetriever, an open-source family of dense text retrievers that combines unsupervised contrastive pre-training with synthetic instruction tuning to outperform much larger biomedical models across 11 standard benchmarks within academic compute budgets.
Developing effective biomedical retrieval models is important for excelling at knowledge-intensive biomedical tasks but still challenging due to the lack of sufficient publicly annotated biomedical data and computational resources. We present BMRetriever, a series of dense retrievers for enhancing biomedical retrieval via unsupervised pre-training on large biomedical corpora, followed by instruction fine-tuning on a combination of labeled datasets and synthetic pairs. Experiments on 5 biomedical tasks across 11 datasets verify BMRetriever's efficacy on various biomedical applications. BMRetriever also exhibits strong parameter efficiency, with the 410M variant outperforming baselines up to 11.7 times larger, and the 2B variant matching the performance of models with over 5B parameters. The training data and model checkpoints are released at https://huggingface.co/BMRetriever to ensure transparency, reproducibility, and application to new domains.
Added
2026-09-26
BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining
Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, Tie-Yan Liu
Why you should read this
Presents BioGPT, a domain-specific generative language model pre-trained on biomedical literature that achieves superior performance on relation extraction and question answering while providing effective text generation capabilities for life sciences research.
Pre-trained language models have attracted increasing attention in the biomedical domain, inspired by their great success in the general natural language domain. Among the two main branches of pre-trained language models in the general language domain, i.e., BERT (and its variants) and GPT (and its variants), the first one has been extensively studied in the biomedical domain, such as BioBERT and PubMedBERT. While they have achieved great success on a variety of discriminative downstream biomedical tasks, the lack of generation ability constrains their application scope. In this paper, we propose BioGPT, a domain-specific generative Transformer language model pre-trained on large scale biomedical literature. We evaluate BioGPT on six biomedical NLP tasks and demonstrate that our model outperforms previous models on most tasks. Especially, we get 44.98%, 38.42% and 40.76% F1 score on BC5CDR, KD-DTI and DDI end-to-end relation extraction tasks respectively, and 78.2% accuracy on PubMedQA, creating a new record. Our case study on text generation further demonstrates the advantage of BioGPT on biomedical literature to generate fluent descriptions for biomedical terms. Code is available at this https URL.
Added
2026-09-24