For audio classification of video data, perhaps you could first extract the audio into wav file and slice them into short audio clips using a tool like FFMPEG with filename corresponding to the time segment of the video. The audio clips can be sent into the network to extract features.