When and by whom the castle was founded is not exactly known, but in 1118 a "Wikker" was mentioned in the records. It is possible that the father of "Eberhard von Wikkeristein" (recorded 1122) had built the castle shortly beforehand. The Wichsensteins were ministeriales of the Bishopric of Bamberg, whose coat of arms bore a blue wolf salient on a silver field. Eberhard was named in 1121 and later as "Eberhard de Lapide", which means "von Stein" or "of the rock". In 1133 the brothers, Eberhard and Wikker de Lapide, were named as joint witnesses in a manuscript from Ensdorf Abbey. Kunstmann was able to ascertain the reason for the change of name was that the home castle of the Wichsensteins, the little castle of Stein, today just a castle site in the village of Stein south of Pegnitz, had to be given up for unknown reasons and they had to built a new castle of their own. In 1165 as well an "Eberhard II de Steine" was appointed as an episcopal "ministerialis", in 1201 Wikker II de Steine was a witness in a deed by Prince Bishop Timo. In 1240 Eberhard III was likewise named as a witness in a document by Frederick III Walpot for Banz Abbey. The first record of Wichsenstein Castle was not, however, until 30 October 1310, when Conrad I of Wichsenstein received 60 pounds of Haller (coins) from the Bishop of Bamberg, Wulfing of Stubenberg and in return had to "guard" the castle for ten years, i.e. fight on the side of the bishop in case of war with his part of the castle and his contingent of troops. In 1328 Bishop Henry II gave Boppo of Wichsenstein a further hundred pounds of Haller for the part of the castle enfeoffed to him. For that he had to concede right of the bishop to buy it back; after that the castle was always a freehold of the Wichsensteins. The Wichsensteins ran into financial difficulties, probably as a result of the economic boom of the towns during the 13th and 14th centuries and the great agricultural crisis of the second half of the 14th century, and became robber barons. George of Wichsenstein was at that time in the service of the brothers Henry and Eberhard of Berg, who had also become robber barons, and was taken prisoner in 1397 by King Wenceslaus following the siege of Spies Castle near Betzenstein.