Tech

Yandex Research Team’s New Compression Methods: AQLM and PV-Tuning

The Yandex Research Team is enhancing efficiency with innovative compression methods such as AQLM and PV-Tuning. In this article, explore the advantages and application areas of these techniques.

Published

2 months ago

July 26, 2024

yönetici

New Compression Methods by the Yandex Research Team

The Yandex Research team introduced two new compression methods at the International Conference on Machine Learning (ICML) ongoing in Vienna, Austria. These methods are called Additive Quantization for Language Models (AQLM) and PV-Tuning. Researchers state that when these methods are combined, they can achieve up to 8 times reduction in model size while maintaining 95% of response quality.

Key Features of AQLM and PV-Tuning

AQLM leverages additive quantization methods traditionally used in information retrieval for compressing large language models (LLMs). This developed method preserves and improves the accuracy of the model under extreme compression, making it possible for LLMs to be widely used on everyday devices like home computers. This results in a significant reduction in memory consumption.

PV-Tuning is designed to correct potential errors that may arise during the model compression process. When AQLM and PV-Tuning are combined, they yield optimal results by providing compact models that can produce high-quality responses even with limited computational resources.

Evaluation of the Methods

The effectiveness of these presented methods has been meticulously tested using popular open-source models such as LLama 2, Mistral, and Mixtral. Researchers evaluated the response quality of these compressed large language models using English comparison metrics like WikiText2 and C4. Despite being compressed by 8 times, the models managed to maintain a response quality of 95%.

Who Can Benefit from AQLM and PV-Tuning?

The Yandex research team emphasizes that the AQLM and PV-Tuning methods will provide significant resource savings for companies developing and distributing proprietary language models and open-source LLMs. For example, the Llama 2 model with 13 billion parameters post-compression can operate using only 1 GPU, leading to up to 8 times reduction in hardware costs. This means that enterprises, individual researchers, and LLM enthusiasts can run advanced LLMs like Llama on their everyday computers.

Exploring New LLM Applications

The researchers state that AQLM and PV-Tuning enable offline distribution on devices with limited computing resources, offering new use cases for devices like smartphones and smart speakers. They express that thanks to advanced LLMs integrated into these devices, users can utilize services like text and image generation, voice assistance, personalized recommendations, and real-time language translation without needing a continuous internet connection.

Application and Access

Today, developers and researchers worldwide can access AQLM and PV-Tuning methods via GitHub. The demo materials provided by developers offer guiding information for effectively training compressed LLMs for various applications. Additionally, it is also possible to download popular open-source models compressed using these methods.

Highlights from ICML

The scientific paper detailing Yandex Research’s AQLM compression method has been published at ICML, one of the most prestigious machine learning conferences worldwide. This study, prepared in collaboration with researchers from IST Austria and experts from the AI venture Neural Magic, demonstrates significant progress in LLM compression technology.

Up Next

Collaboration Between Reddit and Google: Change in the Digital Ecosystem

Don't Miss

New Features in the Google Play Store

Click to comment

Tech

Tesla Semi Fire: NTSB Preliminary Report Released

The NTSB’s preliminary report on the Tesla Semi fire has been released. This content includes important information about the details of the incident, the causes of the fire, and Tesla’s safety measures. Explore the impact on Tesla’s electric vehicles and its future steps.

Published

3 days ago

September 17, 2024

yönetici

The National Transportation Safety Board (NTSB) released a preliminary report stating that a Tesla Semi electric truck in California required firefighters to use “50,000 gallons” (approximately 190,000 liters) of water to extinguish a roadside fire. Firefighting teams also deployed an aircraft to drop fire retardant on nearby areas as a precautionary measure to control the fire.

The accident occurred on August 19 at 3:13 AM local time on the I-80 highway east of Sacramento. The electric truck went off the road while navigating a curve, collided with a roadside boundary marker, and then struck a tree, coming to a stop. Fortunately, the driver was not injured in the accident; however, considering their health condition, they were taken to the hospital.

The Tesla Semi’s large 900 kWh battery caught fire upon impact, and toxic smoke began to spread during the fire. While the fire continued with temperatures reaching up to 538°C, firefighters worked intensively to cool the blaze with water. However, due to the fire’s effects, the vehicle continued to burn until late in the afternoon. Meanwhile, Tesla sent a technical expert to the scene regarding fire safety and high voltage hazards.

The highway reopened to traffic at 7:20 PM local time, 16 hours after the accident. The NTSB, as an organization that can only make recommendations and cannot enforce regulations, sent a team to the area for investigation purposes. This major accident raised several issues, including dangerously hot fires and toxic smoke, which will attract the attention of many international organizations.

The NTSB had previously stated in 2021 that fires in the batteries of electric vehicles pose serious risks to emergency response teams and that the guidelines provided by manufacturers for such fires are inadequate.

Tech

New Leaks and Features About the Samsung Galaxy S25 Ultra

Discover the latest leaks and features about the Samsung Galaxy S25 Ultra. Equipped with new technologies and advanced features, this smartphone elevates the user experience to a new level. Check it out for details!

Published

3 days ago

September 17, 2024

yönetici

New Leaks About Samsung Galaxy S25 Ultra

Samsung is expected to introduce the Galaxy S25 Ultra along with the Galaxy S25 series at the beginning of 2024, likely in January. Although no official announcement has been made yet, leaks regarding the upcoming flagship continue to spread rapidly on the internet. Notable leak source IceUniverse shared several new visuals of the Galaxy S25 Ultra in a post on the X platform. These visuals reveal the design differences between the Galaxy S25 Ultra and the Galaxy S24 Ultra.

One of the notable features in the leaked images of the Galaxy S25 Ultra is the rounded corners of the phone. In contrast, this year’s Galaxy S24 Ultra has quite sharp corners, which has led to feedback from some users indicating discomfort while holding the device. Previous leaks have suggested that the Galaxy S25 Ultra may be somewhat larger.

A tweet shared by IceUniverse: Galaxy S25 Ultra Image

According to leaks, the Galaxy S25 Ultra is expected to be thinner and lighter compared to previous models. In August, rumors emerged that the upcoming Galaxy S25 Ultra would be lighter than its competitors, with a weight of under 221 grams.

Another image featured in IceUniverse’s post shows close-ups of the sides of both next year’s expected flagship phone and Samsung’s current flagship. In this image, the side of the Galaxy S25 Ultra appears flatter with rounded corners. While these changes may not create a major revolution, they will generally positively affect the feel of the phone in hand.

Looking at other rumors about the Samsung Galaxy S25 Ultra, it is expected that the company will offer top-tier screen and camera upgrades next year. Additionally, there is a strong expectation that the satellite connection feature will also be introduced alongside the Galaxy S25.

Tech

iOS 18 Officially Released: Innovations and Features

iOS 18 has been officially released! This update, packed with new features and innovations, elevates the user experience to the next level. Check out our article for details and discover the new offerings of iOS 18!

Published

3 days ago

September 17, 2024

yönetici

iOS 18 Officially Released: Here Are the Innovations!

Apple officially launched iOS 18 yesterday, as promised. After a long beta testing phase, this update brings many exciting new features for iPhone users.

iOS 18 Officially Released: Innovations and Features

New Features in iOS 18

Advanced Customization Options on the Home Screen: With iOS 18, users can now arrange app icons freely without sticking to the traditional grid layout. Additionally, there’s an option to change the colors of the icons and adjust their darkness levels. In short, you are no longer bound to a boring layout. It raises questions why Apple waited until 2024 to offer this feature.
Revamped Control Center: The Control Center is completely redesigned in iOS 18. Users will be able to group controls more comfortably, and developers can easily add controls from their applications. Also, there’s an option to divide the Control Center into pages, allowing different controls on each page.
RCS Messaging Support: With iOS 18, Apple introduces Rich Communication Services (RCS) messaging support. Users can add various effects to their texts and react with new emojis for more colorful responses.
Updated Photos App: The Photos app is presented with a new opening page and a modern layout. Users can now browse photos by time, people, and many other criteria, reorder media files, and explore more with the “Collections” feature.
App Lock Feature: Users can now lock specific applications. For example, they can activate Face ID protection from the menu that appears when long-pressing an app icon. Thus, anyone wanting to open the app will have to pass the Face ID protection first.
Apple Intelligence: One of the important features of iOS 18 is Apple’s artificial intelligence kit, Apple Intelligence. However, to fully benefit from this feature, you’ll need to wait for the iOS 18.1 update. More information about Apple Intelligence can be found here.

How to Install iOS 18?