Welcome to the Research Log, where we share weekly progress on Manifold’s research projects and also the “Pulse of AI”: breakthroughs that are happening in the broader research community we’re excited about!
Interested in contributing or just learning more? Check out the Github or join the conversation on Discord.
NEKO Project
The NEKO Project aims to build the first large scale, Open Source "Generalist" Model, trained on numerous modalities. You can learn more about it here. We’ve made some meaningful progress in the past week:
Text Prediction:
We’ve closed the loop on evaluating our text prediction objective in the NEKO codebase, check out those changes here. Thanks to Bhavul Gauri for leading this work!
Data Sampling:
Our control team has improved sampling performance by nearly 2 orders of magnitude, unlocking huge gains in our ability to train. The new sampling implementation also allows for parallelism over tasks and prefetching data. Implementation to be shared soon! Thanks to Daniel Lawson for making this happen!
AgentForge Project
In the AgentForge project, we’re continuing our survey of various agent approaches, both with LLMs and in other areas of ML. We’re currently reviewing the Gorilla model and paper, and think it is an exciting approach to making progress with tool use models.
Pulse of AI
- One interesting project we have seen is moving from the large amount of online text, image, and video content that has guided generative model training so far to start simulating realistic experiences in response to actions from outside forces (humans, robots, or other). Shoutout to @mengjiao_yang and team for progress on this: https://twitter.com/mengjiao_yang/status/1712153304757915925
- UniSim: Learning Interactive Real-World Simulators, this kind of approach is a key component of scaling robotics: https://universal-simulator.github.io/unisim/
If you want to see more of our updates as we work to explore and advance the field of Intelligent Systems, follow us on Twitter, Linkedin, and Mastodon!
