Testing was carried out largely on real user requests
Yandex specialists compared the quality of responses from the proprietary neural network YandexGPT 2 and GPT 3.5. The study showed that in 63% of cases, Yandex's own language model answers queries better than OpenAI's neural network.
The study was conducted using the Side by Side (SBS) comparison method. Both neural networks answered the same query, and the AI trainers chose the best answer, not knowing which model generated it. The basket of requests was formed from anonymous requests from users of Yandex services to YandexGPT 2. It takes into account the real needs of people and covers many topics.
Technical Director «Yandex Search» Alexey Gusakov said:
Comparisons of neural networks are needed not to obtain abstract numbers in a benchmark, but to solve product problems, create new functions and develop business. That is why a significant part of the comparison basket consisted of real requests from our services.
YandexGPT — a neural network that can create and process texts, offer new ideas and takes into account the context of the conversation with the user. It is available in Alice mode «Let's figure it out». YandexGPT also briefly retells the texts of articles from the Internet in «Yandex Browser» and on 300.ya.ru, summarizes information from product reviews in «Search» and on «Market», helps create product descriptions for sellers of «Market» and not only. The neural network can be tested in your own business — To do this, you need to submit a request for access to the API.