Carnegie Mellon University
CMU Foundation and Language Model Center  ›  Research

Research

CMU FLAME Center is strongly committed to open, responsible, and impactful research. The following is an abridged list of projects carried out by FLAME center faculty. More information is available on the researchers’ web pages.

Note: Most of this research is accompanied by open-source software licensed in ways that make it easy for the research community to use.

Projects from the FLAME Center

12 news articles displayed.

Filter news articles by title

Skip filters and go to news articles.



Reset Filters

LLMs Attack

LLMs undergo extensive fine-tuning to not produce harmful content. LLM Attacks is a proof-of-concept that demonstrates that it is possible to construct adversarial attacks on LLMs in an automated fashion that can bypass this fine-tuning process, with security implementations.

Zeno: AI Evaluation Made Easy

Zeno is an interactive AI evaluation platform for exploring, discovering and reporting the performance of your models. Use it for any task and data type with Zeno's modular views for everything from object detection to audio transcription and chatbot conversations. Zeno helps you move beyond aggregate metrics and spot-checking model outputs to develop a deep and quantitative understanding of how your model behaves.

Machine Learning Compilation for Large Language Models (MLC LLM)

Machine Learning Compilation for Large Language Models (MLC LLM) is a universal solution that allows any language model to be deployed natively on a diverse set of hardware backends and native applications.

FlexFlow Serve: Low-Latency, High-Performance LLM Serving

FlexFlow Serve is an open-source compiler and distributed system for low latency, high performance LLM serving. FlexFlow Serve outperforms existing systems by 1.3-2.0x for single-node, multi-GPU inference and by 1.4-2.4x for multi-node, multi-GPU inference
Visual Generative Pre-Training Score (VisualGPTScore)

Visual Generative Pre-Training Score (VisualGPTScore)

We study generative Vision-language models (VLMs) that are trained for next-word generation given an image. We explore their zero-shot performance on the illustrative task of image-text retrieval. We observe that they can be repurposed for discriminative tasks (such as image-text retrieval) by simply computing VisualGPTScore, the match score of generating a particular text string given an image.
ORCA: Cross-Modal Fine-Tuning

ORCA: Cross-Modal Fine-Tuning

ORCA is a general cross-modal fine-tuning framework that extends the applicability of a single large-scale pretrained model to diverse modalities. ORCA adapts to a target task via an align-then-refine workflow: given the target input, ORCA first learns an embedding network that aligns the embedded feature distribution with the pretraining modality. The pretrained model is then fine-tuned on the embedded data to exploit the knowledge shared across modalities. ORCA obtains state-of-the-art results on 3 benchmarks containing over 60 datasets from 12 modalities.
Sotopia: an Open-ended Social Learning Environment

Sotopia: an Open-ended Social Learning Environment

Sotopia is an open-ended environment to simulate complex social interactions between LLM-based artificial agents and evaluate their social intelligence. Agents role-play and interact under a wide variety of scenarios to achieve complex social goals.
WebArena: A Realistic Web Environment for Building Autonomous Agents

WebArena: A Realistic Web Environment for Building Autonomous Agents

WebArena is a standalone, self-hostable web environment for building autonomous agents. WebArena creates websites from four popular categories with functionality and data mimicking their real-world equivalents. This is accompanied by a benchmark on interpreting high-level realistic natural language command to concrete web-based interactions.
Cross-modal few-shot adaptation with CLIP

Cross-modal few-shot adaptation with CLIP

A simple cross-modal adaptation approach that learns from few-shot examples spanning different modalities. We demonstrate that one can indeed build a better visual dog classifier by reading about dogs and listening to them bark. To do so, we exploit the fact that recent multimodal foundation models such as CLIP are inherently cross-modal, mapping different modalities to the same representation space.
Generating Images with Large Language Models (GILL)

Generating Images with Large Language Models (GILL)

GILL is capable of processing arbitrarily interleaved image-and-text inputs to generate text, retrieve images, and generate novel images. GILL is the first approach capable of conditioning on arbitrarily interleaved image and text inputs to generate coherent image (and text) outputs.
Prompt2Model - Generate Deployable Models from Instructions

Prompt2Model - Generate Deployable Models from Instructions

Prompt2Model is a system that takes a natural language task description (like the prompts used for LLMs such as ChatGPT) to train a small special-purpose model that is conducive for deployment.
Load more articles