In mobile devices there exist several in-built sensor units and sources which provide data for context reasoning. More context sources can be attached via wireless network connections. Usually, the mobile devices and the context sources are battery powered and their computational and space resources are limited. This sets special requirements for the context recognition algorithms. In this paper, several classification and automatic feature selection algorithms are compared in the context recognition domain. The main goal of this study is to investigate how much advantage can be achieved by using sophisticated and complex classification methods compared with a simple method that can easily be implemented in mobile devices. The main result is that even a simple linear classification algorithm can achieve a reasonably good accuracy if the features calculated from raw data are selected in a suitable way. Usually context recognition algorithms are fitted to a particular problem instance in an off-line manner and modifying methods for on-line learning is difficult or impossible. An on-line version of the Minimum-distance classifier is presented in this paper and it is justified that it leads to considerably higher classification accuracies compared with the static off-line version of the algorithm. Moreover, we report superior performance for the Minimum-distance classifier compared to other classifiers from the view point of computational load and power consumption of a smart phone.