It is actually a hostility to Berbers and Turks that explains their being blamed for the decadence of Islam. The Seljuk role, it could be said, was in actual fact instrumental in saving whole Islamic domains from total extinction.
Summarised extracts from a full article:
The Question is...? Myths and Fallacies Surrounding the Decline of Muslim Civilisation by Salah Zaimeche
It has been noted by several Western historians such as Shaw that:
[Islam] could not rule since the barbarians (Turks, Berbers, etc.) took over the lead of Islam. The Islamic world then entered in a period of ignorant brutality, from which it emerged only to fall into the mournful agony in which it is struggling at present.
Wiet also paints a picture of the Muslim lands suffering at the hands of these forces
In the eleventh century the Moslem world was subjected to major invasions, those of the Berbers and especially of the Turks.
Upon closer inspection, we find, it is actually a hostility to Berbers and Turks that explains their being blamed for the decadence of Islam. This hostility has its reasons. The Berbers, both Almoravids and Almohads, have contributed to a major historical phase of Islamic history, that is, holding up the Christian crusade advance in Spain. Indeed, soon after the death of the leader al-Mansur, Muslim Spain fell into disunity and chaos during the era of the `party (taifa) kings' (reyes de taifas, muluk at-tawa'if) (1009-1091), when the Peninsula dissolved into as many as thirty more or less independent rulers, who fought each other. This emboldened Christian princes in North West Spain who gradually moved south, absorbing one Islamic region after the other, very often using one against the other. In their ultimate panic some Reyes called on the Almoravid leader, Yusuf Ibn Tashfin, to assist them on three occasions; but each time after crushing the Christian armies, he was asked to leave Spain, to be re-called once the Reyes (taifa rulers) were threatened again. The third time he was invited, in 1090, Ibn Tashfin crossed the straits of Gibraltar from Morocco, and this time eliminated the inept Reyes, and installed Almoravid rule all over the country. Under Almoravid rule not only was the unity of the Muslim Peninsula regained, but also there re-appeared in the West a combative form of Islam that responded to the Christian combativeness.
When the Almoravid power collapsed, the Almohads came to the fore in 1147. Their most determinant victory was on the 18th of July 1196, when they inflicted a crushing defeat on the Christian alliance of many armies at Alarcos, the Christian army being virtually exterminated. However, once their rule was weakened by internal rivalries, the Almohads were themselves crushed at Las Navas de Tolosa in 1212, a defeat which Lewis rightly recognised, `broke the back of Muslim power in the Peninsula.Cordova fell in 1236, Seville in 1248, and soon followed the other towns and cities, only leaving the Grenada enclave which was to fall in 1492. The important outcome, however, is that had the Berbers failed to respond to the Spanish advance over two centuries, it would have been very likely that Christian Spain would have proceeded to occupy the whole of North Africa .
Similar grounds with respect to the hostility to the Seljuks apply. In 1092, the leading vizier of Seljuk history, Nizam al-Mulk, was murdered. He was the founder of the madrassa system, which taught that no government could be secure without a religious base.A month after Nizam's death, Malik Shah (1072-92), the third Seljuk sultan, died in suspicious circumstances, followed closely by his wife, grandson and many other powerful political figures. Leaderless, the Islamic world was rife with intense internal strife while jockeying for power in both the Eastern Islamic world and Egypt ensued. The two sons of Malik Shah, Barkyaruk and Muhammad were locked in a conflict, which ate up all the available military resources throughout the East. In Syria, there were also small, mutually hostile city states. Eleventh century Syria, Lamonte notes, was `a crazy quilt of semi independent states.In the middle of this Muslim disunity, Pope Urban II (1088-1099) launched his call for the Crusades. His stated reasons for attack, as he said in his rallying call to the Crusades:
`It is urgent for you to bring hastily to your brothers in the Orient the help so often promised and that's of pressing necessity. The Turks and the Arabs have attacked them...., and penetrating always further inside Christian countries, have on seven occasions beaten them in battle, and have killed and taken captive a great number, and have destroyed churches and devastated the kingdom.'
However, Hillenbrand notes, the timing of the First Crusade simply could not have been `more propitious. Had the Europeans somehow been briefed that this was the perfect moment to pounce? Unfortunately there is little evidence on this in the Islamic sources, but seldom had the arm of coincidence been longer.
The only force that stood to fight was that of the Seljuks. The Seljuks, under the leadership of Qilij Arslan, Mawdud, Il-Bursuqi and others, fought the Crusaders until the rise of the first leader who was able to unite the Muslim armies: Imad Eddin Zangi (himself a Seljuk) (1127-1146), followed by his son Nur Eddin (Zangi) (1146-1174). Thus, in the crucial period when the Muslims were at their most divided, it was Seljuk opposition that managed to limit the impact of the Crusades, and inflict on them the biggest reverses such as that at Nicea in October 1096, when the Seljuk Turks led by Qilij Arslan destroyed the Crusade vanguard and the whole of its advanced camp. In 1099, the Turks crushed the `Peoples Crusade' at Civetot (Anatolia); in the year 1100, the Seljuk Danishmand fell on Bohemund near the town of Malatia, and after decimating his army took Bohemund prisoner. In 1104, the Franks, leaving Edessa in search of plunder within Muslim towns in the vicinity, were met (by the river) by the Turks who decapitated the Frankish army, and took into captivity Baldwin of Edessa and Count Jocelyn of Tell Bashir. In April 1110, the Atabeg Mawdud, the Seljuk commander of Mosul, began moving against Edessa with the support of the Ortoqid Ilghazi and the Emir of Mayyafaraqin, Soqman al-Qutbi. Informed by their spies, the Franks hastened to meet him at the Euphrates. The Franks, although more numerous, were defeated by the Turks.Ghazi, an Ortoqid Turk, met by the combined forces of Roger of Antioch, Baldwin of Jerusalem and Galeran, was able to beat them at the battle of Balat (also known as the field of blood) in June 1119. Aleppo was on the verge of starvation when it was rescued by Il-Bursuqi, governor of Mosul in January 1225, forcing the Franks into retreat. When Il Bursuqi was murdered in 1127, Imad Eddin Zangi was appointed to replace him as commander of the east. After conquering the citadel of al-Atharib in 1130, with his legendary fierceness, Zangi destroyed it and razed it to the ground. Under Zangi's command a serious mobilisation of jihad began. Greater and more dedicated forces coalesced around him which in 1137 led to the first Muslim successes in recovering Kafartab, Maarat al-Numan, Bizaa, and Athareb from the Franks. Zangi led his offensive against Edessa, and for the first time since the Crusaders' arrival in 1096, Islamic forces were united around him: Turks, Arabs and Kurds fighting together. The taking of Edessa in 1144 marked a significant turning point in Muslim fortunes, for not just the city, but the whole state was regained for Islam; by far the greatest victory of the Muslim forces for nearly half a century. So distraught was the Pope with such loss, that the second Crusade was launched shortly afterwards. 
The Seljuk role, it could thus be said, was in actual fact instrumental in saving whole Islamic domains from total extinction particularly in regard to the wholesale slaughter by the crusaders of populations in Antioch, Maarat al-Numan and Jerusalem
To read about the other reasons thought to be behind the decline of Muslim Civilisation and a discussion of the various theories propounded by western historians, read the full article linked above.
 E. Renan: Averroes et l'Averroisme, p. iii.
 G. Wiet et al. p.7,
 For details on the rule of al-Mansur and the break up of the kingdom see S.P. Scott: History ; op cit;.
 S.P. Scott: History of the Moorish Empire; op cit; Vol 1; p.453 fwd.
C. Cahen: Orient et Occident au temps des Croisades, Aubier Montaigne, 1983. p.21.
John Glubb: A Short History of the Arab Peoples; Hodder and Stoughton, 1969. p.190.
B. Lewis: Cultures in Conflict; Oxford University Press; 1995. p. 19.
 W. Durant: The Age of faith, op cit; Chapter XIV; p.309.
C.Hillenbrand: The Crusades, Islamic Perspectives, Edinburgh University Press; 1999. p.33.
 C.Hillenbrand: The Crusades, Islamic Perspectives, op cit;.p.38.
J.H. Lamonte: crusade and Jihad: in N.A. Faris ed. The Arab Heritage, Princeton University Press, 1944. pp 159-198; p.163.
 Discours of Pope Urban II. It is re-transcribed by Foucher of Chartres in Regine Pernoud: Les Hommes et la Croisade, Jules Tallandier, Paris, 1982.
 C.Hillenbrand: The Crusades, Islamic Perspectives, op cit;.p.33.
R. Finucane: Soldiers of the Faith; J.M. Dent and Sons Ltd; London, 1983. p.21.
 The first and second Crusades; op cit;.p.74.
The first and second Crusades from an Anonymous Syriac Chronicle: Translated by A.S. Tritton; with notes by H.A.R. Gibb: pp 69-101. Journal of The Royal Asiatic Society (JRAS) 1933.p.79.
 S.Runciman: A history of the Crusades, Cambridge University Press; 1951;Vol ii, p. 115.
 The first and second Crusades; op cit; p.83.
 C.Hillenbrand: The Crusades, Islamic Perspectives, op cit;.p.21.
 The first and second Crusades.p.96.
 Ibn al-Furat: Tarikh al-Duwal wal Muluk; ed. M. F. El-Shayyal; unpublished Ph.d.; University of Edinburgh; 1986. IV; p. 30.
 S.Runciman: A History; op cit; Vol ii; pp. 219-20.
 The First and Second Crusades: Part two: April. pp 273-305 p.280.
 C.Hillenbrand: The Crusades, Islamic Perspectives, op cit;.p.112.
 See appropriate sections in S.Runciman: A History; op cit; G.Lebon: La civilization des Arabes; op cit; etc