On the 6th of December, Sundar Pichai unveiled Gemini, an LLM boasting multimodal reasoning capabilities, heralded as "the most powerful and versatile model ever constructed" by Google. The first iteration of Google response to GPT-4 comes in three sizes: Gemini Nano, Gemini Pro, and Gemini Ultra. According to a document shared by Dylan Roussel on X, Google is on the verge of disclosing the new moniker for its chatbot, along with Gemini Advanced, fueled by Gemini Ultra 1.0.
For a quick recap, the Nano version caters to Android phones, and an enhanced version of Gemini Pro amplifies Bard's functionalities, particularly in reasoning, comprehension, planning, summarization, and coding. Initially accessible in English across 170 countries and territories, Google recently announced its global availability wherever Google Bard is deployed, with language support encompassing French.
While initially slated for an early 2024 release, Gemini Ultra's performance metrics were shared by Google.
Scoring at 90.0%, compared to GPT-4 86.4%, Gemini Ultra achieves a milestone as the first model to surpass human experts in Massive Multitask Language Understanding (MMLU). This involves a fusion of 57 subjects, including mathematics, physics, history, law, medicine, and ethics, testing both global knowledge and problem-solving prowess.
Google conducted a comparative analysis between Gemini Ultra and GPT-4 across various benchmarks, such as code and reasoning, with Gemini Ultra outshining GPT-4 in most aspects, except for common-sense inference in HellaSwag.
Anticipated Google Modifications: As per the shared document, initially dated February 7th but revised to February 8th, the forthcoming modifications to the Bard/Gemini chatbot aim to enhance user-friendliness, reduce visual distractions, and streamline navigation.
Gemini Advanced: Positioned as a premium service, Gemini Advanced will grant subscribers access to additional functionalities through Gemini Ultra 1.0. According to Google, Gemini Advanced is poised to handle highly intricate tasks, including coding, logical reasoning, nuanced instruction tracking, and creative collaboration.
Although the document does not specify subscription costs, it mentions availability in over 150 countries and territories, supporting multiple languages but being fully optimized only for English queries.