Learn
MedArena: LLM Arena for Clinicians
MedArena is a free platform for clinicians to compare different large language models (LLMs) in an unbiased, head-to-head competition. Similar to other leaderboards, such as those from Huggingface, MedArena is a platform to compare LLMs for different tasks using the Elo rating system to rank models based on human preferences.
For human evaluation of LLM outputs, HumanELY was created based on comprehensive, comparable, and quantifiable metrics using Likert scale.
Connect
The BrainX Community Live, January 2025 event, featured Suhana Bedi, Stanford University; Dr. Ashish Atreja, UC Davis; VALIDAI & Dr. Yanshan Wang, University of Pittsburgh. The session was moderated by Dr. Piyush Mathur, Cleveland Clinic; BrainX. At the panel discussion, key aspects of human evaluation of LLMs in healthcare, such as metrics for evaluation, evaluation methods, including who and how these need to be performed, and frameworks, were reviewed. Future directions, such as new frameworks, tools, and the possibility of LLMs-as-judge, were also explored.
Datasets
DiagSet: a dataset for prostate cancer histopathological image classification
The dataset consists of three different partitions: DiagSet-A, containing over 2.6 million tissue patches extracted from 430 fully annotated scans; DiagSet-B, containing 4675 scans with assigned binary diagnosis; and DiagSet-C, containing 46 scans with diagnosis given independently by a group of histopathologists.
Related Publication: Koziarski, M., Cyganek, B., Niedziela, P. et al. DiagSet: a dataset for prostate cancer histopathological image classification. Sci Rep14, 6780 (2024). https://doi.org/10.1038/s41598-024-52183-4
Conferences
Additional BXC-featured publications
General/ Generative AI/ LLM
Multimodal Medical Code Tokenizer
General/ Generative AI
The TRIPOD-LLM reporting guideline for studies using large language models
General
Join and follow the BrainX community!
Webpage: https://brainxai.org/
Newsletter: https://brainxai.substack.com/subscribe
LinkedIn: https://www.linkedin.com/groups/13599549/
Youtube: https://www.youtube.com/channel/UCua5EiLL6I29hpNrJsdv1rg