The oldest surviving written grammar for any language is believed to be the Tolkāppiyam (தொல்காப்பியம்), a book on the grammar of the Tamil language, written around 200 BC by Tolkāppiyar. Its classification of the alphabet into consonants and vowel was a breakthrough. The historical record of the study of language begins in North India with Pāṇini, the 5th century BC grammarian who formulated 3,959 rules of Sanskrit morphology, known as the Aṣṭādhyāyī (अष्टाध्यायी). Pāṇini’s grammar is highly systematized and technical. Inherent in its analytic approach are the concepts of the phoneme, the morpheme, and the root; the phoneme was only recognized by Western linguists some two millennia later.

In the Middle East, the Persian linguist Sibawayh made a detailed and professional description of Arabic in 760 CE in his monumental work, Al-kitab fi al-nahw (الكتاب في النحو, The Book on Grammar), bringing many linguistic aspects of language to light. In his book he distinguished phonetics from phonology.

Later in the West, the success of science, mathematics, and other formal systems in the 20th century led many to attempt a formalization of the study of language as a "semantic code". This resulted in the academic discipline of linguistics, the founding of which is attributed to Ferdinand de Saussure.

Languages are not just sets of symbols. They also often conform to a rough grammar, or system of rules, used to manipulate the symbols. While a set of symbols may be used for expression or communication, it is primitive and relatively unexpressive, because there are no clear or regular relationships between the symbols.

For example, imagine going on a walk with a person who only knows individual symbols. If you saw a dog, he might say, "Dog not scare" or "Not scare dog". Although any English speaker would have some notion of what he was talking about, the relationship between the words is unclear. Is he scared of dogs? Or just that dog? Or does he want to scare the dog off? Does he think the dog is scared? But if you respond, "I'm not scared of dogs", the relationship between "dog" and "scare" is quite apparent and hence the meaning of the utterance.

Another property of language is the arbitrariness of the symbols. Any symbol can be mapped onto any concept (or even onto one of the rules of the grammar). For instance, there is nothing about the Spanish word nada itself that forces Spanish speakers to use it to mean "nothing". That is the meaning all Spanish speakers have memorized for that sound pattern. But for Croatian speakers nada means "hope".

However, it must be understood that just because in principle the symbols are arbitrary does not mean that a language cannot have symbols that are iconic of what they stand for. Words such as "meow" sound similar to what they represent (see Onomatopoeia), but they could be replaced with words such as "jarn", and as long as everyone memorized the new word, the same concepts could be expressed with it.