One of my favourite journals is Hakira which often contains excellent scholarly articles on a wide range of subjects. Volume 7 contains a fascinating article about the history of how we light the Hanukkah candles entitled: The order of lighting the Hanukkah candles: The Evolution of a Custom and the Influence of the Publication of the Shulhan Arukh by J. Jean Ajdler. Here is a link to the whole article.
I will just quote from the conclusion which is fascinating.
“The order of the lighting the Hanukkah candles was not raised until the thirteenth century. In this era the customs of R. Meir of Rothenburg (c.1215-1293) and R. Jonah Gerondi (c.1200-1263) were recorded by their disciples.
From two responsa of the fourteenth century we know about two other customs; the Austrian custom and the custom of Maharik corresponding in all likelihood, to the old French custom. Separate traditions evolved from amongst the three main strands of Ashkenazi Jewry, the Rhineland, the French and the Austrian (and East European). Apparently the Spanish tradition was similar to that of the Rhineland.
It is well known that the French tradition which was most influential during the twelfth and thirteenth centuries under the leadership of the French Tossafists lost its influence under the persecutions and the expulsions of the Jews of England91 in 1290, of the Jews of France in 1306 and the final expulsion of the Jews of Provence in 1498. The departure of R. Joseph Colon to Italy is connected to the worsening conditions of the Jews of Savoy during the fifteenth century. The French tradition remained alive in the small communities of Northern Italy (Piedmont and Lombardy) and it disappeared slowly, first by assimilation to the mainstream and then with the disappearance of the small communities in the nineteenth century following industrialization and the rural exodus.
R. Joseph Caro made extensive use of the responsa of Maharik in his Beit Yoseph, and in this case of the order of lighting of the Hanukkah candles he accepted his ruling. The success and the increasing influence of Shulhan Arukh, made it possible for this ruling to gain popularity and enabled this custom to spread quickly. It was adopted immediately into the Sephardic world, and more slowly but steadily into the Ashkenazi world.
During the sixteenth and seventeenth centuries new lighting schemes were developed in order to satisfy contradictory halachic requirements. Today the ruling of the Shulhan Arukh is practically universally accepted with only very few, small communities behaving differently, e.g. Perushim of Jerusalem who still follow the Rhineland custom of the Gra. This evolution of customs may seem surprising to a society which is not at all accustomed to changes and evolution. In our everyday life, we have the impression that Jewish traditions are immutable. Even in issues where an adaptation is desired by a significantpart of the community, change seems impossible and scholarly rabbinical initiatives remain theoretical, without practical consequence.
Nevertheless given the breadth of Jewish history we must acknowledge that changes occur periodically without us really knowing exactly when and how. An example of one such change is expressed by one the great medieval Provencal rabbis, R. Zerahia ha-Levi. He writes: “And do not be astonished about what I said, that the custom (of the blowing of the shofar) changed in later generations… I remember myself that I saw in my youth that the whole community was praying the Amidah of Mussaf of Rosh ha-Shannah and recited only seven blessings and only the chazan recited nine blessings. They were relying on the custom of the two Babylonian Yeshivot of the Gaonim because they found this written in their books. You also find this old custom in the halakhot of R. Isaac ben Giyat. But today we see that every one prays nine blessings”. Similarly he writes: “and so did all the generations that were preceding us (they observed only one day Rosh ha-Shanah in Israel). But now, recently, sages of Provence arrived there and taught them to celebrate two days of Rosh ha-Shanah according to the requirements of the Halakhot of the Rif.”