Research Log #020

Santiago Pedroza -

Welcome to Research Log #020! We document weekly research progress across the various initiatives in the Manifold Research Group, and highlight breakthroughs from the broader research community we think are interesting!

We’re growing our core team and pursuing new projects. If you’re interested in working together, join the conversation on Discord and check out our Github.

NEKO

The NEKO Project aims to build the first large scale, Open Source "Generalist" Model, trained on numerous modalities including control and robotics tasks. You can learn more about it here. 

  • Language: We are currently aiming to achieve a level of performance in line with GPT-2 across the language modality. The team is using the Open Webtext dataset to perform test runs, if you are interested in seeing our progress or contributing to the project, check out the Github branch here!
  • Datasets: Along the Control task modality, the survey for viable datasets continues. We’re delving deeper into possible alternatives to the GATO proprietary training data. Check out the datasets we’ve surveyed so far here, and let us know what other datasets may be helpful for the project!

Pulse of AI

  • Scaling deep learning for materials discovery: Material discovery can be time consuming and resource intensive, and due to the combinatorial nature of chemicals, random exploration can be computationally intractable. Recent rapid advances in deep learning have made exploring the space of physically possible materials tractable using predictive methods. Using deep learning, Merchant et. al have found “an order-of-magnitude expansion in stable materials known to humanity”. They have found 2.2 million new structures, and currently replicated 736 new materials. More can be read through their paper in Nature.
  • Generating Multi-View Optical Illusions with Diffusion Models: It is now possible to generate optical Illusions using diffusion models. By averaging noise from different perspectives, images can be generated which create multiple possible interpretations. If you want to dive deeper and have your mind melted, check their website.

If you want to see more of our updates as we work to explore and advance the field of Intelligent Systems, follow us on Twitter, Linkedin, and Mastodon!