LearnLM, Google’s gen-AI model fine-tuned for learning, outperforms many leading AI systems in adhering to the principles of learning Science. In learning science includes elements like explaining concepts at appropriate levels, providing effective hints and guidance, interactively guiding learners, and encouraging active engagement.  

Outperformer 

In the evaluations against contemporaneous flagship models, each representing a company’s premier offering as of 2024-10-01, expert pedagogical raters preferred LearnLM with an average preference strength of 31% over GPT-4o, 11% over Claude 3.5 Sonnet, and 13% over the original Gemini 1.5 Pro. 

Preferred over others 

This report demonstrated that LearnLM excelled in following instructions, supporting learning objectives, and adapting to learners' abilities. Pedagogy experts and testers consistently preferred LearnLM's results over those of other models.

An organic approach to learning 

 An impressive example of its capabilities includes tasking it with teaching complex topics, which it simplifies into manageable chunks and employs guiding questions to foster independent research rather than offering direct answers. This resembles a more organic and interactive approach to learning a particular concept rather than mugging things and making a uni-lateral conversation. 

Data and analysis 

The experiment yielded a dataset comprising 2,360 conversations, totaling 58,459 messages exchanged between learners and models. Additionally, 10,192 expert assessments of these conversations were collected, with an average of three experts reviewing each conversation pair.

Experts expressed a preference for LearnLM over the competing model more frequently, often citing themes such as keeps_on_topic, challenges_learner, and gives_away_answers. Conversely, when the other model was preferred, the explanations tended to highlight themes like clarity, info_amount, and conversation_style.