Exposure of humans to unusual spaces is effective to observe the adaptive strategy for an environment. Though adaptation to such spaces has been typically tested with vision, little has been examined about adaptation to left–right reversed audition, partially due to the apparatus for adaptation. Thus, it is unclear if the adaptive effects reach early auditory processing. Here, we constructed a left–right reversed stereophonic system using only wearable devices and asked two participants to wear it for 4 weeks. Every week, the magnetoencephalographic responses were measured under the selective reaction time task, where they immediately distinguished between sounds delivered to either the left or the right ear with the index finger on the compatible or incompatible side. The constructed system showed high performance in sound localization and achieved gradual reduction of a feeling of strangeness. The N1m intensities for the response-compatible sounds tended to be larger than those for the response-incompatible sounds until the third week but decreased on the fourth week, which correlated with the initially shorter and longer reaction times for the compatible and incompatible conditions, respectively. In the second week, disruption of the auditory-motor connectivity was observed with the largest N1m intensities and the longest reaction times, irrespective of compatibility. In conclusion, we successfully produced a high-quality space of left–right reversed audition using our system. The results suggest that a 4-week exposure to the reversed audition causes optimization of the auditory-motor coordination according to the new rule, which eventually results in the modulation of early auditory processing.