Research Log #010

Harsh Sikka -

Starting 09/21, we're starting to share The Research Log via our shiny new blog (which you are currently on if you're reading this!).

The Research Log is basically a weekly update on all the progress we're making in building next generation Intelligent Systems, and we also share what we've been finding interesting in the broader research community in the "Pulse of AI" section that follows.

Glad to have you here, let's dive in!

NEKO Project

The NEKO Project aims to build the first large scale, Open Source "Generalist" Model, trained on numerous modalities. You can learn more about it here.

Text Modality Training is working

The Text Modality thrust, led by Bhavul Gauri, has resulted in successful training of the NEKO model on small sets of text data! Our architecture is officially multimodal.

GitHub - ManifoldRG/gato-control at add_text_modality
Gato implementation for control tasks to be used for Manifold’s Neko, or other projects. - GitHub - ManifoldRG/gato-control at add_text_modality
GitHubManifoldRG

Parallel Sampling in NEKO

The Control team, led by Daniel Lawson been implementing parallel sampling techniques in NEKO to train on more data in less time. Checkout our progress here:

GitHub - ManifoldRG/gato-control at parallel_sampling
Gato implementation for control tasks to be used for Manifold’s Neko, or other projects. - GitHub - ManifoldRG/gato-control at parallel_sampling
GitHubManifoldRG

AgentForge Project

The AgentForge Project aims to build models, tools, and frameworks that allow anyone to build much more powerful AI agents capable of using tools and interacting with the digital and physical worlds.

AI Agents Survey

Our open source, distributed survey is underway. Check out the progress below and consider contributing your understanding of AI Agent related papers.

AF Survey - Review Material
Research Papers Review Material,Tag,Type,PoC,Status,Why is it relevant? (1-2 lines, or just few words),Rate out of 4 (Novelty, Citations, Depth, Breadth),Github Issue,PR to Manifold-KB,(Temp Column) Takeaways Doc Test Row EXAMPLE ROW,Survey Review Material,Research Paper,<a href=“mailto:bhavul9…
Google Docs

SotA AI Agents Exploration

We're currently exploring new conceptual agents that push SotA AI Agent capabilities forward. Stay tuned for more updates!

Pulse of AI

DALLE-3 is coming out! It is using GPT-4, which means OpenAI is strongly pushing towards multimodality.

DALL·E 3
DALL·E 3 understands significantly more nuance and detail than our previous systems, allowing you to easily translate your ideas into exceptionally accurate images.

We're reviewing a few interesting papers this week, check them out!

Chain-of-Verification Reduces Hallucination in Large Language Models
Generation of plausible yet incorrect factual information, termed hallucination, is an unsolved issue in large language models. We study the ability of language models to deliberate on the responses they give in order to correct their mistakes. We develop the Chain-of-Verification (CoVe) method wher…
arXiv.orgShehzaad Dhuliawala
Multimodal Foundation Models: From Specialists to General-Purpose Assistants
This paper presents a comprehensive survey of the taxonomy and evolution of multimodal foundation models that demonstrate vision and vision-language capabilities, focusing on the transition from specialist models to general-purpose assistants. The research landscape encompasses five core topics, cat…
arXiv.orgChunyuan Li