11. Given the function $f(x)=\frac{\sqrt{3}}{2} \sin 2 x-\frac{1}{2}\left(\cos ^{2} x-\sin ^{2} x\right)-1, x \in \mathbf{R}$, after shifting the function $f(x)$ to the left by $\frac{\pi}{6}$ units, we get the function $g(x)$. Let $\triangle A B C$ have angles $A, B, C$ with opposite sides $a, b, c$ respectively.
(1) If $c=\sqrt{7}, f(C)=0, \sin B=3 \sin A$, find the values of $a, b$;
(2) If $g(B)=0, \vec{m}=(\cos A, \cos B), \vec{n}=(1, \sin A-\cos A \tan B)$, find the range of $\vec{m} \cdot \vec{n}$.
(1) $f(x)=\frac{\sqrt{3}}{2} \sin 2 x-\frac{1}{2}\left(\cos ^{2} x-\sin ^{2} x\right)-1=\frac{\sqrt{3}}{2} \sin 2 x-\frac{1}{2} \cos 2 x-1=\sin \left(2 x-\frac{\pi}{6}\right)-1$. $f(C)=\sin \left(2 C-\frac{\pi}{6}\right)-1=0$, so $\sin \left(2 C-\frac{\pi}{6}\right)=1$.
Since $2 C-\frac{\pi}{6} \in\left(-\frac{\pi}{6}, \frac{11}{6}\right)$, then $2 C-\frac{\pi}{6}=\frac{\pi}{2}$, so $C=\frac{\pi}{3}$.
By the cosine rule: $a^{2}+b^{2}-2 a b \cos \frac{\pi}{3}=7$, and since $\sin B=3 \sin A$, by the sine rule we know: $b=3 a$; solving gives: $a=1, b=3$;
(2) $g(x)=\sin \left(2 x+\frac{\pi}{6}\right)-1$, so $g(B)=\sin \left(2 B+\frac{\pi}{6}\right)-1=0$, so $\sin \left(2 B+\frac{\pi}{6}\right)=1$.

Since $2 B+\frac{\pi}{6} \in\left(\frac{\pi}{6}, \frac{13 \pi}{6}\right)$, then $2 B+\frac{\pi}{6}=\frac{\pi}{2}$, i.e., $B=\frac{\pi}{6}, \vec{m}=\left(\cos A, \frac{\sqrt{3}}{2}\right), \vec{n}=\left(1, \sin A-\frac{\sqrt{3}}{3} \cos A\right)$,
thus
$$
\vec{m} \cdot \vec{n}=\cos A+\frac{\sqrt{3}}{2}\left(\sin A-\frac{\sqrt{3}}{3} \cos A\right)=\frac{1}{2} \cos A+\frac{\sqrt{3}}{2} \sin A=\sin \left(A+\frac{\pi}{6}\right)
$$

Since $B=\frac{\pi}{6}$, then $A \in\left(0, \frac{5}{6} \pi\right)$, so $A+\frac{\pi}{6} \in\left(\frac{\pi}{6}, \pi\right)$, thus $\sin \left(A+\frac{\pi}{6}\right) \in(0,1]$, i.e., $\vec{m} \cdot \vec{n} \in(0,1]$.