1) In the davening of Rosh Hashanah [and Yom Kippur] we say many lofty piyutim (liturgies) which were composed by extremely holy individuals (Tanaim, Gaonim and even Rishonim the likes of Rashi and the Baalei Tosafos).
Just to give you an idea of the holiness we are talking about here, the Magen Avraham (Siman 68) quotes the Ariza"l who says that when Rav Elazar HaKalir (who was either a Tana or a Rishon, and one of the preeminent writers of our Kinus, Selichos and Piyutim) was composing a certain piyut for the Musaf of Rosh Hashanah (that begins "V’Chayos Asher Heina M’Rubaos Kisei etc.") a ring of [heavenly] fire encircled his entire body.
Many of these piyutim are written in very complex language (done so purposely, according to the Sh"ut Shaar Ephraim, so the non Jews wouldnt "Steal" them and use them in their prayes in their Bais Avoda Zara), and comprised with references to many sources throughout the revealed and hidden Torah. (In fact in over 50 places, the Baalei Tosafos bring proofs to their opinions from stanzas in these piyutim!)
Therefore, the average person will have a very hard time understanding the piyutim and Tefilos of the Yamim Noraim unless they spend some time beforehand studying and analyzing them, which is exactly what the Poskim say each Jew should do a few days before Rosh Hashanah. (Today many Machzorim make it a lot easier to do this, as they have translations and/or explanations on the page. However, it is best to read them before Rosh Hashana if possible, and not during the actual davening. See also Mishna Berura Siman 100 in the name of the Ta"z)
2) It is also important to learn the Halachos of and meaning behind Tekias Shofar prior to Rosh Hashanah, as well as to learn Sifrei Mussar which will cause a person’s heart to "wake up" and fear Hashem, as well as bring a person to grasp the reality of Hashem’s greatness as He is about to judge all of mankind (Kitzur Shulchan Aruch) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~