The mp3 History

Early 1970s: Prof. Dieter Seitzer of Erlangen-Nuremberg University in Germany begins wrestling with the problem of transmitting speech in high quality over phone lines. Although a first patent application was rejected, he establishes a group of students interested in audio coding research who start tackling the problem for their diploma and Ph.D. theses.

Late 1970s: With the introduction of ISDN and fiber optic cables for telecommunication, improved speech coding seems to become less important. As a new idea, the team of Prof. Seitzer starts research in coding of music signals.

1979: Prof. Seitzer's team develops the first digital signal processor capable of audio compression. During subsequent development, Karlheinz Brandenburg, a student in that team, starts employing psychoacoustic principles in the audio coding schemes, feeding back important findings and corrections into the science of the hearing properties of the human ear. Guided by Prof. Seitzer, Brandenburg and the team achieve continuous improvement of their coding algorithms.

1981: The Compact Disc is presented to the public. For storing audio data, it uses uncompressed linear PCM with 16 bit/sample.

A research alliance is formed between Erlangen-Nuremberg University and the Fraunhofer Institute for Integrated Circuits within the framework of the European Union funded EUREKA project EU147 for Digital Audio Broadcasting (DAB). Led by Prof. Heinz Gerhäuser (Fraunhofer IIS), the joint research team puts up a functional real-time codec of the LC-ATC (Low Complexity Adaptive Transform Coding) algorithm as the basis for its research. Up to that point, LC-ATC existed only as a simulation on computer. Due to the enormous computing time required it could be tested only with a very limited amount of audio material. The real-time codec enables testing LC-ATC under real-world conditions and leads to significant algorithmic optimizations. The hardware system comprising multiple Digital Signal Processor (DSP) modules and a number of audio and data I/O interface cards is developed by a group of scientists including Harald Popp and Ernst Eberlein.

The real-time implementation of the OCF algorithm in 1989 was one of the most important milestones in the development of mp3.

The Moving Picture Experts Group MPEG is established as a working group of the International Standardisation Organisation ISO. It is in charge of developing compressed digital audio and video standards.

Brandenburg finishes his doctoral thesis on the OCF algorithm (Optimum Coding in the Frequency Domain), a codec exhibiting a number of characteristics of the eventual mp3 technology, like a high frequency resolution filter bank, non-uniform quantization, Huffman coding, and its side information structure. The software part of the real-time system for OCF is mainly developed by Bernhard Grill under the lead of Prof. Gerhäuser. In this phase, the basic OCF technology is extended towards a system that is capable of coding audio signals at 64 kBit/s in good quality for the first time worldwide. That way, transmission of music in real-time over telephone lines could be achieved.

The real-time implementation of the OCF algorithm in 1989 was one of the most important milestones in the development of mp3.

1989 OCF is suggested as the MPEG audio standard.

As the first client of Fraunhofer IIS »Christian Science Publishing Society CSPS« in Boston uses OCF to transmit their radio program.

A new high-performance audio codec called ASPEC (Adaptive Spectral Perceptual Entropy Coding) is presented as the result of further improvements of OCF and contributions by Hannover University, AT&T, and Thomson. The development partners had proposed ASPEC already in 1989 for the forthcoming MPEG audio standard.

MPEG in total received 14 proposals for audio coding. Encouraged by MPEG, the audio coding contributions are merged into four proposals including ASPEC and MUSICAM. After several formal tests, MPEG suggests erecting a family of three audio coding schemes from MUSICAM and ASPEC: Layer 1 is a low complexity variant of MUSICAM, Layer 2 an optimized version of MUSICAM and Layer 3 is based on ASPEC.

Due to its lower complexity, Layer 2 is selected by DAB as the audio format for digital audio broadcasting services whereas the ASPEC / Layer 3 coder with its highest coding efficiency known at that time (and higher complexity) becomes the key to transmitting high quality audio via ISDN phone lines. As a proof of concept, Fraunhofer IIS manufactures and sells a number of ASPEC 19” studio racks to professional users such as radio stations. In this way, transmitting speech and music reliably via ISDN between broadcasting studios becomes the first real application of the Fraunhofer IIS audio coding algorithms. The evolution from ASPEC to the final "mp3" (MPEG-1, Layer 3) codec involves some technological harmonization with the other planned MPEG-1 audio coders and the addition of a joint stereo coding mode, the latter developed by Jürgen Herre. In December, the technical development of the MPEG-1 standard is finished. Layer 3 offers near CD quality at less than 2 bits/sample.

MPEG concludes its first compression standard, MPEG-1, for use in video CDs (CD-I); the first edition is released as ISO/IEC 11172 in August 1993. In its audio section, a generic family of three codec formats (Layer 1, 2, 3) is specified. Being the most efficient of the three codecs, Layer 3 soon becomes a popular way to store music on the relatively small hard disk drives of PCs and to transfer music files over the Internet through 28.8kbps PC modems.

The Olympic winter games in Albertville are broadcast via ASPEC by all German private stations.

The first generation of one-chip mp3 decoders by Micronas are introduced in a Solid State Player. Shortly after, Fraunhofer IIS develops the prototype of an mp3 player. This first portable music player without moveable parts saves about a minute of an mp3 track and proves the suitability for daily use.

The Fraunhofer-Gesellschaft and Thomson establish a joint license program. This way, users of MPEG Audio Layer 3 can quickly and easily receive software implementations and usage rights for patents of Fraunhofer IIS and Thomson. Already in 1992, directly after the standardization, the first companies license mp3.

The name “mp3” is coined in 1995. In an internal poll, Fraunhofer researchers unanimously vote for .mp3 as the file-name extension for MPEG Audio Layer 3. In the same year, Fraunhofer provides a first PC based Layer 3 codec as shareware.

Fraunhofer IIS starts with the sales of the mp3 software over the Internet. Shortly after, an Australian student buys the software via a stolen credit card number and makes it publicly available. The stolen software spread quickly worldwide.

In the same year, Worldspace Radio chooses mp3 for audio coding.

Michael Robertson starts mp3.com as a web site for information about mp3 technology, encoder, player, and anything else related to mp3. Not long after that, mp3.com was the biggest legal mp3 music sharing service.

The era of portable mp3 listening begins with the introduction of Diamond Multimedia’s "Rio 100" in the U.S. and Saehan Information Systems' "MPMAN" in Korea. They are the first portable players using solid-state flash memory to store and play compressed mp3 music files either downloaded from the Internet or encoded from a music CD.

Fraunhofer IIS participates in the »Secure Digital Music Initiative«, which was founded by the music industry to avoid music piracy through copy protection.

Karlheinz Brandenburg, Bernhard Grill and Harald Popp are awarded with »Deutscher Zukunftspreis« as representatives for the entire development team.

In the U.S., the first portable audio devices equipped with hard drives and the first headphone CD players with mp3 functionalities are brought to market. One of them was the HanGo player. Thanks to the built-in 5 GB laptop hard drive used as a storage medium, the player could store up to 80 hours of music, making it the first device capable of holding an entire music collection. The MP3 decoder was developed by Fraunhofer IIS and ran on a Motorola DSP.

mp3 increasingly becomes a cultural phenomenon, too, with hundreds of millions of mp3 capable computers and consumer electronic devices sold annually.

Fraunhofer IIS and Thomson introduce mp3 surround, an mp3 upgrade for high quality 5.1 surround sound. The data rates of mp3 surround are comparable to those of stereo mp3. mp3 surround is compatible with every mp3 end device and software.

In Germany, mp3 creates more than 10,000 jobs and contributes 300 Million Euros to Germany’s annual tax revenue from mp3 players and mp3-related products.

The possibility to store thousands of mp3 encoded songs on a small portable player, search them by album, artist, title, genre or even to have play lists generated automatically has reawakened millions of people's love for music. You no longer have to look for a misplaced CD in order to listen to a song you have missed for years. The whole music collection is available at the push of a button. In fact, inexpensive 40GB mp3 players today can store over 10,000 songs of CD quality readily available whenever and wherever you are.

mp3 is more than just a technology. It is a sensational development that connects musicians with music lovers, speakers with their listeners, artists with their audience.

The increasing popularity of mp3 portables triggers dozens of companies to launch similar portable devices for compressed music playback, and stimulates the development of additional audio codecs for use in PCs and consumer electronics.

In 2007, Fraunhofer IIS celebrates 20 successful years of developing audio coding algorithms. Today, the German institute is the worldwide leading independent research lab in the field of high quality audio compression technologies for the modern media world.

In May 1987, developers at Fraunhofer IIS achieved a first important milestone: They presented a real-time implementation of the “Low Complexity Adaptive Transform Codec”, in short LC-ATC. That way, music could be compressed in real-time for the first time – a huge advantage as before compression of audio signals took hours even using high performance computers. Preliminary work started already in the 70s with the goal of compressing music for transmission over phone lines.

World-renowned formats such as mp3 or MPEG-4 AAC evolved from the early beginnings. Thanks to these developments, Fraunhofer IIS enjoys an excellent reputation as the leading independent research lab in the field of audio coding.

Germans spend around 1.5 billion Euros on mp3 players and mp3 related products.

Technicolor introduces the result of another collaboration with Fraunhofer IIS: mp3HD, the solution for lossless audio coding. mp3HD is compatible with every mp3 end device.

At the beginning of the 2010 semester, an unprecedented cooperation began between Friedrich-Alexander Universität Erlangen-Nürnberg and Fraunhofer IIS: The International Audio Laboratories Erlangen took up their work.

To sustainably secure the leading position Fraunhofer IIS had achieved in audio coding, the institute reinvested about €60 million of the licensing revenue from the mp3 patents.

For their significant contributions to the development and implementation of the global mp3 standard, Prof. Dr.-Ing. Dr. h.c. Karlheinz Brandenburg, Prof. Dr.-Ing. Bernhard Grill, and Prof. Dr.-Ing. Jürgen Herre receive the 2015 Technology Award of the Eduard Rhein Foundation at Deutsches Museum Munich on October 17.

The book “The mp3 Story” is published. The book by former Fraunhofer Society spokesman Franz Miller describes the long journey from the initial idea to the global success of the technology.

T-Mobile USA becomes the first mobile network operator to introduce the new EVS mobile codec.

Mp3 was the first generation of audio codecs "made in Erlangen", and today, their fourth generation is well established on the global market. Discover several important milestones of this journey.

The mp3 licensing program by Technicolor and Fraunhofer ends and the last core mp3 patent included in it expires.

MPEG-H Audio becomes part of the ATSC standard and is used in 24/7 regular broadcasting in Korea.

xHE-AAC becomes part of the Android operating system; Apple follows in 2019 and Microsoft in 2020.

The new Bluetooth Low Energy standard LE Audio is released and includes LC3 as mandatory audio codec.

The fifth generation of audio codecs achieves bitrates of 3 kbit/s or less for speech signals. This unprecedented efficiency is made possible by the smart application of AI-based methods. One of the new codecs can be used, for example, in mobile satellite communication.
AI is also used increasingly for audio processing such as the noise suppression of upHear Microphone Processing, which ensures clear listening experiences – for instance on a smartphone – and extends battery life with its low computational requirements.

Nevertheless, researchers are not only focused on working with audio signals: a completely new field of activity opens up with work on generative models and Large Language Models (LLMs). An early result was Teuken-7B, an efficiently trained European language model that was not only trained in the 24 official languages of the European Union but also complies with EU data protection standards.
In the future, Teuken-7B and other language models are to be adapted for use in public administration, the judiciary, and industry. To support this, AI compute capacity is currently expanded together with Friedrich-Alexander Universität Erlangen-Nürnberg.